DOCUMENT RESUMiE 



ED 282 G61 



CE 047 275 



AUTHOR 
TITLE 

INSTITUTION 

SPONS AGENGY 

PUB DATE 
NOTE 

PUB TYPE 

EDPS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



Riccio, James; And Others 

Vi rginia Employment Services Program, Final 
Report. 

Manpower Demonstration Research Corp., New York, 
N.Y^ 

Ford Foundation, New York, N.Y. ; Virginia State Dept, 
6£ Social Services , Richmond . 
Aug 86 
298p. 

Reports - Evaluative/Feasibility (142) 
MF01/Pei2 Plus Postage. 

Adults; Career Education ; *eost Effectiveness ; 
Demonstration Programs ; Economically pi sad van t aged; 
*Employment Services; Federal Legislation; Federal 
Programs; Federal State Relationship; * Job Search 
Methods ; *Part icipat ion ; *Program Effectiveness ; 
Program Evaluation; State Programs; Statewide 
Planning; *Welf are Recipients 

Impact Studies; *Virginia; Work incentive Program 



Thii report updates the interim Virginia Employment 
Services Program (ESP) part icipat ion patterns and examines program 
success in mandating job search participation. Following ah 
introductory chapter^ chapter 2 discusses research design, sample, 
and data sources. (The sample contained 1,287 Work Incentive Program 
applicants and 1,897 recipients .) Chajpter 3 discusses the nature of 
EiSP activities and extent of enrbllee participation^ It analyzes 
cumulative participation patterns, presents a detailed assessment of 
program success in reaching its targeted caseload,, describes 
activities in which enrollees participated, considers participation 
of important subgroups within the target population, and examines 
participation patternsover time. Chapter 4 describes short- and 
longer-term impacts. It shows mo^ gains in employment for the 
experimental group inthe short run and employment impacts for the 
£ull_ sample almostenti rely from stronger gains for appl icants , not 
recipients. Applicants in particular show longer-term program effects 
that continue the storyof the fuli_ sample ' s short-term gains^_ 
Chapter 5 analyzes ESP costs and benefits. It indicates that ESP 
enrollees, and applicants in particular^ had increased earnings that 
exceeded their increased tax payments as well as their losses in 
welfare and in payments from other transfer programs . Benefits to the 
government budget are also shown. Supplementary tables are appended. 
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This is the second and final report on the Employsient Services Program 
(ESP) in Virginia* a statewide initiative intended to improve the 
employment prospects of people receiving public assistance- The ESP study 
is part of HDRC»s larger multi-state Demonstration of State Work/Welfare 
initiatives. Other states in this project include Arkansas, California, 
Illinois, Maine, Maryland, New Jersey, and West Virginia. 

The Demonstration of State Work/Welfare Initiatives has been a unique 
opportunity for MDRC to work closely with a number of states to evaluate 
their employment programs. At the same time, MDRC has been able to examine 
a subject of national as well as state cdhcern: the critical relationship 
between work and welfare dependency. Addressing state issues in a manner 
that benefits policy at many levels is a challenge that MDRC has been 
privileged to undertake. 

Siis dononstration also documents an important shift in program 
responsibility away from the federal government to the states. The studies 
evaluate the initiatives states themselves chose to implement under the 
provisions of the Cinibus Budget Reconciliation Act of 1981 * in which they 
received authority for the first time to operate Community Work Experience 
Programs (GWEP) for recipients of Aid to Families with Dependent Children 
UFDC) and to streamline the administratidn of their Work Incentive (WIN) 
systems. Because states responded to these options in different ways^ the 
dononstratibn is not built around a single model. Rather, the initiatives 
represent some of the major variations being tried in this country and span 



a range of local economic conditions and AFDC program provisions. 

Most states receive two research reports over the course of the demon- 
stration. The first report for Virginia covered issues of implementation 
and early participation. This second and final report updates that 
information and presents the impacts of ESP services on enrollees' 
employment and welfare receipt as well as the costs and benefits of the 
program. 

MDRC could hot have conducted this demonstration without the support 
of The Ford Foundation^ which provided funds for the planning stage and for 
the evaluation activities of the participating states, matching an equal 
investment of state or other local resources. Siis joint funding relation- 
ship is another significant aspect of the demonstration. 

Throughout the course of the Demonstration of State Work/Welfare 
initiatives, MDPC has been gratified the sustained commitment of the 
participating states and foundations and their interest in the findings, 
it is our hope that the process arid results of this demonstration have 
contributed to informed decisibh-making and will ultimately lead to the 
deveio^ent and operation of even more effective programs designed to 
increase the self-sufficiency of welfare recipients. 

Barbara B. Blum 
President 
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EXECUTIVE SUMMARY 



This is the second and final report on an evaluation of the 
ebmionweafth of Virginia's Employment Services Program (ESP), a statewide 
initiative designed to improve the employment prospects of individuals 
applying for or receiving welfare under the Aid to Families with Dependent 
Children (AFDC) program. The multi-component ESP has as its principal 
focus job search assistance — sometimes group, but primarily individual — 
followed by work experience, education or training. Job search is required 
of all enrollees at program entry and periodically thereafter. Services 
following job search are not universally mandatory; 

This evaluation of the program was conducted between August 1983 and 
August of this year by the Manpower Demonstration Research Corporation 
(MDRC) under a contract from the state's welfare agency, the Department of 
Social Services (DSS). Because Virginia's AFDC system is state-supervised, 
but locally administered, the state's 124 local welfare agencies have been 
responsible for the operations of ESP. Eleven agencies participated in the 
research, and they were considered roughly representative of the agencies 
statewide. 

While all 11 local agencies successfully implemented the job search 
requirement for the female WIN-mandatory population targeted, participation 
in post- job-search services was more limited, partly because of resource 
constraints. Agencies themselves could not easily operate educational or 
training ccmponents, so they mostly referred enrollees to existing 
vocational and school programs, in which other residents of the community 
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could enroll; 

Overall, the sttady found that ESP led to modest employment gains and 
welfare savings. This was particularly true for the women who came into 
the program at the time they were applying for welfare (i.e., the welfare 
applicant group in the research), for whom program effects, or impacts^ 
tended to increase over ah 1 8-month follow-up period. However, it was not 
yet clenr if these impacts would continue to grow over time, stabilize or 
decline. For the recipient group — or women already receiving welTare 
at program entry immediate iinprbvemehts in employment and welfare 

savings were short-lived. The program, in addition, did not reduce the 
size of the caseload for either group; there was no Increase in case 
closings. 

In weighing the economic benefits and costs of ESP, the results were 
generally positive. Over a five-year period (which inciucJes both observed 
and extrapolated estimates) women assigned to the program, and applicants 
in particular, were better off as a result of the program to the extent 
that their increased earnings and fringe benefits exceeded their increased 
tax payments and losses in welfare and payments from other transfer 
programs. The government budget also benefited. Over the same five-year 
period, the estimated increase in tax revenues and savings from welfare and 
other programs exceeded the costs of ESP services for the applicant group. 
For the recipient group^ budget savings and program costs werr roughly 
equal* 

The magnitude of the findings in Virginia fall within the range of the 
results found in studies of other programs included in MlDRC's Demonstration 
of State Work/Welfare Initiatives, of which Virginia is parti This demon- 
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stration is a large-scale muiti-year study of a variety of eBQployment 
initiatives for the welfare pbpuiation developed by 11 states. Although 
the treatments and characteristics of the target population vary in each 
state^ this studies to date have found modest employment gains in all states 
arid welfare savings in some. 

The results in Virginia deserve special attention because the state 
chose to target all of it? WIN-mandatory caseload, including hew applicants 
and the longer-term, more deperident recipient group. (Those classified 
WIN-mandatory are usually parents of school-age children and are thus 
required to register for program services. The WIN-mandatory caseload in 
Virginia is about one- third of the total AFDC welfare caseload.) State 
resources are thus spread over a Lroad section of the welfare population, 
not just the applicants or the small group of newly determined mandatory 
recipients, the target populations of some of the other state programs in 
the MDRC demonstration. 

Given this focus and the reliability of these study results, obtained 
from a carefully executed research design^ the ESP evaluation in Virginia 
offers an unusual opportunity. The study is pble to assess the effects and 
potential of an empldyment program directed to a population that has many 
labor market deficiencies, but represents a cross-section of the WIN- 
mandatory caseload. 

Background 

ESP was developed in response to the federal Omnibus Budget 
Seconoili-tion Act (OBRA) of 1981, which permitted states to restructure 
the Work Incentive (WIN) Program, the major federally- funded employment and 
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training prbgrair for the AFDG welfare population. Under the authority 
granted by OBRA's WIN Demonstration Program title, Virginia began ESP in 
1983f placing the overall management responsibility in the state's welfare 
agency, the Department of Social Services. 

According to Virginia's ESP regulations, local welfare agencies are 
required to provide job search assistance to eligible AFDC applicants and 
recipients, and to condition the receipt of welfare on participation in 
this activity. Moreover, the agencies are required under a State Statute 
of I982 to operate a work experience component and offer "opportunities" 
for education and training, although they can set their own criteria for 
assignment. Individuals who are not required to register with ESP ~ 
including mothers with children under five years of age are encouraged 
to volunteer for the program. The 11 local agencies selected for ra)RC»s 
evaluation represent both urban and rural areas of the state. Together 
they cover approximately one-quarter of the state's ESP registrants. 

According to the state-designed program model, the principal services 
were to be: 

• Job Search Asststanoe: An initial Job search requirement to 
last for up_ to four weeks, and to be repeated every six 
monthsi Agencies could decide whether to offer group job 
search workshops (a fairly structured activity) or individual 
self-directed job search in which, after limited guidance^ 
enrbllees were required to contact a specified number of 
potential employers ^ 

♦ Work Exp erience.^ _An unpaid work position in a public or 
private nonprofit agency for up to 13 weeks, aithoitgh reassign- 
ment was possible for an additional 13-week period. The compo- 
nent was a v^riation^ of the Community Hork Experience Program 
(eWEP)j^ authorized by OBSA, in which enrollees are required to 
workinreturn for their welfare benefits. The number of work 
hours in Virgijtia ±s_ determined by dividing the monthly grant 
aiBount by the minimum vage^ although a maximum was established 
of 86 hours per month. 
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• Edud&tion and Tra i ^i^fi g S e rv ^eea . Provided mostly by referral 
to existing coinmunlty vocatlbhal programs or schools that were 
either free >^ low-cost^ or funded by other resources including 
JTPA, Pell Grants or the like^ Examples of such activities 
include community college eburaes; basic education and GED 
preparatibn classes^ usually offered by public schools; 
training programs operated by outside providers under the Job 
Training Partnership Act (JTPA); and privately operated 
training programs. 

In addition, enrollees were to be assessed again after each program 
activity or at least every three months, except for women employed 
part-time or enrolled in long-term education or training, who were assessed 
every six months. Throughout their program tenure * enrollees also received 
chiid-care support and transportation expenses, if these were necessary. 

This ifinal report updates some of the findings from the interim 
report, which examined ESP's implementation and early participation. This 
report's primary focus, however, is the program impact and benefit-cost 
analyses, presented here for the first time. 

the Stud y Design and Research Sample 

The study used an experimental research design, according to which 
female AFDC applicants and recipients were randomly assigned to either an 
experimental group, required to enter ESP^ or to a control group. Members 
of the control group received no ESP services, but could, on their own, 
enter other programs in their communities. The research sample was formed 
over a period of 14 months, with random assignment extending from August 
1983 through September 1984. 

As noted above, the sample consisted of a cross-section of the female 
WIN-mandatory caseload in Virginia. In addition to AFDC applicants, it 
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included riecipients who had been registered with WIN for some time, as well 
as those who had just become WIN-maridatbry^ typically because their 
youngest child had become five. Applicants made up about iiO percent of the 
total research sample of 3|18^; recipients^ 60 percent. 

At the time they entered the study, enrdllees were primarily single 
heads of households with an average of two children. Two-thirds were 
nonwhite, and 80 percent were, between 25 and years old. these women had 
substantial barriers to employment. Over half were high school dropouts. 
About 88 percent had been on welfare before, and about half had not worked 
in the two years prior to sample entry. 

• A substantial jproportibn of experiment ais took_ part ^1^ ESP 
activities. Within nine months of random assignment, 58 
percent entered at least one program activity^ usually 
individual job search. 

As anticipated, job search — the only required activity for all 
enrollees was the most widely used ESiP service. Slightly more than half 
of all experimentals took part in it — he percent in individual job search 
and 15 percent in group workshops. While individual job search offered 
less assistance than group search, its flexibility and the fact that it did 
not require daily participation allowed a fairly substantial proportion of 
People to be involved in the program. 

In addition^ all applicants were required to take part in a separate 
"applicant job search" immediately after welfare application. Proof of 
three employer contacts was a condition of welfare approval. While most 
women satisfied this requirement, almost none found jobs through this 
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minimal job search effdrti 

• Erirollees were much less active in post- job search services. 
During a hihe-mbnth follbw-up period ^ about 10 percent of the experi- 

inehtals took part in work experlehce, some more than once. Approximately 
12 percent enrolled in education courses or vocational programs^ Education 
was twice as frequently assigned as training, particularly in the urban 
areas. 

• For the relatively small number who participated in work 
experience^ the reaction was generally positive, although the 
positions did hot usually help to build new skills* 

In Virginia^ as in five other MDRC demonstration states that offered a 
structured wor): experience cbmjpoheht^ personal interviews were conducted 
with participants and their worksite supervisors to determine the quality 
of work and participants* attitudes toward the positions. A total of il7 
participants and supervisors were interviewed in Virginia, and their 
responses are analyzed in the final report. 

Generally, most participants said that they liked the jobs, and both 
they and their supervisors perceived the work as necessary and not 
"make-wdrk." However, the jobs mostly entry-level clerical^ maintenance 
and food service pbsitibris required few skills. As a result, they did 
not usually help participants tb develbp skills that they did hot already 
possess. 

• Erirpllees were mbstly referred tb coBmunlty _ education and 
training programs. In which anyone whb qualified cbuld enroll. 
ESP did not increase erirbllmerit ih such prcgrams over the 
level of participation that would have occurred In its 
absence. 

the expected difference between the experi.Tierital and control grbujp in 
participating in education and training did not materialize: both grbups 



had roughly similar levels of participatibh. This was partly due to the 
agencies* lack of resources to provide these services on their own, as well 
as their difficulty in forging new linkages with outside providers, 
especially in the early period of the Job Training Partnership Act. 

Consequently, the impact analysis compares two groups that differed in 
the receipt of other services (i.e., job search and wort, experience), but 
not in the level of education and training. Program impacts thus cannot be 
directly attributed to the inclusion of education and training services in 
the program model; 

• Fewer applicants than recipients participated in ESP 

activities. 

Recipients were considerably more likely than applicants to parti- 
cipate in the program's activities. Overall, 69 percent of recipients 
became active In at least one component^ compared to only k2 percent of 
applicants. The different rates are partly explained by the higher 
deregistration, or departure rates of applicants from the program (55 
percent) compared to recipients (3^1 percent). About one-third of the 
applicants were denied welfare and never received program services, while 
others left welfare or the program for other reasons. 

• Qnly a very small proportion of enroilees remained continu- 
ously enrolled in ESP without fulfilling the program's 
requirements. 

Although a substantial proportion of enroilees did not participate in 
ESP activities, many were not subject to the program's requirements for 
very long. Some left welfare or the program shortly after entering: for 
example^ because of remarriage^ employment, an increase in other income, 
illness or other reasons. These departures limited the piroportion of 
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people with whom staff cbuid work during any given period. Other 
enrbllees^ both applicants ai:d recipients, fulfilled the program's 
requirements by working, although their low incomes allowed them to remain 
on welfare with continued ESP registration if their employment was less 
than 30 hours per week. 

When all of these factors are taken into account, the program's 
ability to reach the targeted caseload is more evident. At the ninth month 
after random assignment, 9^1 percent of the applicants and 89 percent of the 
recipients had satisfied the program's requirements through either partici- 
pation or employment, or had been deregistered. (Some portion of the 
uricbvered group was in a "pending status," with the possibility of later 
participation.) ESP thus succeeded in implementing a participation 
requirement as a condition for receiving welfare, although it was mainly 
limited to one cbmpbheht job search. 





The impact analysis examined the effectiveness of ESP in improving 
experimentals' employment arid reducing their use of welfare. The impacts 
were estimated by comparing the outcomes of the experimental group with 
those of the control group over a uriiform period of time. While not all 
experimentals participated in ESP activities, averages calculated for the 
experimerital group include butcomes for all nonpartlcipants as well as 
participants. (This is because it is impossible to identify within the 
control group a subgroup of those who would have participated had they had 
this opportunity. A comparisbri of experimerital participants to all 
controls could yield biased results.) Tests of statistical significance 
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indicated how likely it was that measured differences resulted from the 
program treatment rather than from chance. It is important to note that 
even small impacts can be important if they are long-lasting or if they 
have occurred for a large number of people. 

impacts are first presented for the full sample, in which the 
foiiow-up period was a relatively short nine months (i.e., three quarters) 
for employment and earnings and one year for welfare receipt and payments. 
Applicant and recipient subgroups are next examined separately for both the 
short-term period and over the longer term as welii The subsampie used to 
examine longer-term program effects was the early part of the full sample, 
those randomly assigned from August 1983 through March 1984, for whom 18 
months of AFDC follow-up records were available, and 15 months of data on 
employment and earnings. 

Employment and earnings data were obtained from records in the 
Virginia Unemployment Insurance System, and AFDC welfare records were the 
source of monthly welfare grants. Since the two systems organize their 
data somewhat differently, this accounts for the different lengths of 
follow-up, as noted above. 

The importance of having a control group in this kind of a study 
should be clearly understood. Because of random assignment, the two groups 
are similar in all respects except receipt of ESP services; cbnsequent- 
ly^ the control group represents what would have happened to experimental 
members had the program not existed. The outcomes of the control group 
serve as a benchmark against which the outcomes of the experimental group 
can be judged. 

This is particularly important because many people do not stay on 
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welfare for long. Welfare is often a temporary source of income for many 
who wiii find jobs on their own or (leave welfare for other reasons. Among 
members of the control group in this stJdy, ^1 percent found employment 
within nine months, even without ESP assistance. Over time, the average 
earnings of this control group increased, while their welfare grant 
declined. (See table 1;) 

fmpaots for the Fall SMPle 

• PY?r^l^» ESP led to jDOdest increases in employment and earn- 
ings. While the employment gain was statistically signifi- 
canty the earnings gain was not. 

Daring the th:^ee follow-up quarters, the employment rate of the experi- 
mental group increased by 3; 3 percentage points over the ko.5 percent level 
of the control group. (See Table 1.) the average amount experimentals 
earned during the follow-up period increased by $81, an improvement of 8 
percent from the control group average of $1,038; While the employment 
gain was statistically significant, the earnings impact was net. 

• ESP reduced the amount of welftire benefits paid to the 
experimental group, but did not substantially lower the 
propor»tion of individuals receiving welfare; 

For the full sample, ESP generated an $84 reduction in the average 

amount of AFDC payments received by the experimental group during the 

one-year follbw-up period. This is a statistically significant U percent 

reduction from the control group average of $2^007. It should be noted 

that this finding represents a redtactton in the average grant level of 

experimentals compared to the average grant for controls, and not an 

increase in case closings. Although the proportion of persons ori welfare 

declined over time within the experimental group, the same pattern was 
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TABLE 1 



SUMMARY OF PROGRAM IMPACTS FOR THE FULL SAMPLE 



Outcome and Fol Low-Up Period 


1 

Experimentale 


Co n t r oL 6 


01 f f e rence 


Ever EmpLbyisd, Quarters S - 4 (X] 


43 .6 


40.5 


+ 3.3* 


Ever Employed [%] 

Quarter of Random ABeignment 
Quarter 2 
Quarter 3 
Querter 4 


27.2 
28.3 
31.2 
34,4 


25.7 
26.4 
27.9 
3D. 5 


+ 1 .5 
+ 1,9 
+ 3.3*» 
+ 3.9** 


Mvoia||H lUbDL CBrningBy uUBrbBrB c*^** i*j 


1119.05 


1 038.16 


+80.89 


Averisge Totel Ee rni nge [$ ] 

Querter of Rendom AEslgnment 
Querter S 
Qiia r te r 3 
Que r te r 4 


221.29 
284.71 
380 .65 
453.70 


224.22 
284.64 
3 46 .1 4 
407.38 


- 2.93 
+ 0,06 
+ 34.51 
+ 46.32 


Eve r Race 1 ved Any AFDC Pey me h t 

UUBrbCrE 1 4 i To } 


86 .0 


86 .1 


- 0,1 


Ever Recolved Any AFDC PeymehtG; [%] 
Quarter of Random Aeelgnment 
Querter 2 
Qua r ta r 3 
Que r t i3 r 4 


82. 8 
76.3 

65^9 
59;7 


62.9 
76.4 
67.5 
59.8 


- 0.2 
-0.0 

- 1 .6 

- 0.1 


Average Total AFOC Peymehte 
Recalvedy Querters 1-4 ($] 


1 923^28 


2006^87 


-83 .59** 


Average AFDC PeymentB Received [t] 
Quarter of Random ABeignment 
Qua rt er 2 
Quarter 3 
Qua r t a r 4 


542^23 
522.52 
447 ;95 
410.58 


551 .47 
546 .71 
478.31 
43 0,3 8 


- 9i24 
-24.1 9* 
-30i36** 
-1 9,80 



SOURCE: See Table 4.2. 



NOTESt Enptoymant and earnings impaste cover a period of nine months 
beginning with the quarter after the quarter of rendom eeaiUnment, Welfere impecte 
cover a t«atve nonth period Including the qoarter of random aseighmeht. There may be 
Bome di acrepanci 88 In calculating a xpe r i me n t a l-co n t r o I differencee due to rbUnding. 



A tNO- tailed t-teet wae applied to experimentel-control differences, 
Statiaticat significance levels are indicated es: * = 10 percent| ** = 5 percent; *** 
- 1 percent. 
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observed for the control group. In other woi^ds, similar proportions of the 
experimental and control groups were on the AFDC rolls throughout this 
period; 

Impacts A g.-^^ ADDliGants 

• For applicants to welfare, ESP led to gradually increasing 
employBent gains. T*e iargest gain occurred during a longer- 
term foliow-up of 15 monthr. 

A3 seen in Table 2, the employment rate for applicants improved by 5.4 

percentage points over the short-term, a 10 percent gain over the control 

rate of 5^.1 percent. These impacts grew larger over time. Tor a group of 

early applicants who were followed for 15 months, employment rates were a 

statistically significant 7.5 percentage points higher than the control 

level of 37.4 percent at the end of the longer-term follow-up. (See Figure 

1.) 

The earnings gain of $127 for the full applicant sample in the short 

term was not statistically significant. However, for the early applicants 

tracked for 15 months, the earnings impact became significant in the sixth 

quarter. The increase of $166 over the control group mean of $525 

represents a 32 percent improvement. 

• There is also evidence that over time, produced welfare 

savings for J:he aj)pl leant grqup._ The prograia also appeared to 
reduce the propoption_ of applicants receiving welfare. Both 
measures were stattsttcaily significant by the end of an 
18-month follow-up period. 

For the full appitcaht group, the welfare savings of $75 was not 

statistically significant in the one-year follow-up period. The evidence 

In Table 2 also indicates that the program requirements did not deter 

people from continuing to seek welfare: similar proportions of both 
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T^LE 2 

aJMMARY OF mCBRAM IMPACTS TOR THE RJLL ^AMPLE, BY WaFARE STATIJS 



I 



Outcome And FoLLor-Up Period 



AppL leant 6 



Ex perlweht elB Control 8 D1 f f erence 



Recipients 



Experlmentete Controte Difference 



Ever EBployed^ Quertere 2 - A [%] 
Ever Bnployed [%] 



59.5 



54.1 



+ 5.4* 



33^4 



31.1 



+ 2.S 



Querter of Random Aeslgnment 


45.7 


47.1 


- 1.4 


14.7 


11 i3 


+ 3.4** 


GiMrter 2 


•♦1 mO 


3S.3 


+ 2.0 


18.7 


17i4 


+ 2.3 


Querter 3 


44.0 




T D.J** 




20^9 


+ 1 ;9 


Quarter 4 


46.8 


39.6 




OR n 

CD .U 


24. 2 


+ 1 08 


Average Totel Eernlnge, 














Quarters 2^4 1%) 


1631.87 


1504.57 


+127.30 


776.12 


713.82 


+ 62i30 


Average Totel EernlngE ($] 














Quarter of Random ABBlgnment 


3S3.13 


443.34 


-50,20 


103 .66 


77.51 


+26 ;1 5 


Quarter 2 


423 ;SS 


Af\0 "k; Q 
**Oti .Da 


~dB.37 


191 .82 


168.41 


+23i41 


Quarter 3 


561 .89 


492 .70 


' u 0.10 


cD u . 0 0 


c«41 .Bb 


+18i72 


Quarter 4 


646.76 


559.28 


+87 .47 


«9c«9 ./c 


303 .55 


+20 i1 8 


Ever Received Any AFDC Payment^ 














Quarters 1-4 {%] 


69.3 


69.2 


+ 0.1 


97.2 


97.8 


- 0i6 


Ever Received Any AFDC Peymente [%] 














Querter of Random Aeelgnment 


61.8 


61.3 


+ 0.4 


95.9 


97.6 


- 0i7 


Quarter 2 


58.6 


56.2 


+ 2.4 


88.2 


S0.3 


- 2i1 


Quarter 3 


44.7 


45.7 


- 1.0 


80.1 


82.4 


- 2.3 


Querter 4 


37.8 


40.2 


- 2.4 


74.4 


73.2 


+ 1.2 


Averege Total AFDC Peymente 














Received^ Quarters 1 ^ 4 ($j 


1S0S.46 


1277.68 


-75.22 


2407.90 


2508.79 


-1 00. 99** 


Averege AFDC Peymente Received ($] 














Querter of Random Aeelgnment 


315.79 


314.51 


+ 1.28 


695.30 


712.83 


-17.53 


Querter 2 


369.74 


364.69 


-14.95 


625.18 


658.36 


-33.1 8*» 


Querter 3 


274.91 


307.77 


-32.86 


563.95 


596.32 


-32. 37** 


Querter 4 


242.02 


270.70 


-28.89 


523.36 


541 .28 


-17.92 



SOURCE: 



See Tables 4^3 and 4*5^ 



NOTES: Employment and •o'^nlnge Impacts cover a period of nine mdnthe beginning with the querter 
after the quarter of random aeelgnnanti Wetfera Impacts cover a ^elve mbnei period 1nc:lud1nB the querter 
of random eselgnment. There mey be some d1 scr epanci ee In catcotatlng experlmentel-cbntrol differences due 
to rounding. 



A two-telled t-test was applied to experimental-control differences. Btetlstlcal significance 
levels ere Indicated as: ♦ = 10 percent; = 5 percent; and = l percent. 
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FIGURE \ 
VIRGINIA 



AFDC APPLICANTSr TRENDS IN C3UARfERL.Y EMPLOYMENT 
RATES ANO AVERAGE AFDC PAYMENTS 

(EARLY SAMPLE) 
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experimentals and cbntrbis received welfare st some period during the year. 

In the longer-term^ however^ impacts appeared to become stronger. 
Welfare savingsi or grant reductions, for the early applicant sample 
continued their short-term growth into the 1 8-month foilow-up period, the 
average cumulati.e savings of $232 in this period was statistically 
Significant, as was a reduction in the proportion of experimentals on the 
weirare rolls. The proportion dropped in the last three months by 8.4 
percentage points^ down from the control group level of 40.8 percent. 

Impacts Amo ng R e cibi ents 

• For AFDC recipients^ ESP produced little change in employment 
and earnings. 

The full recipient sample experienced a small but not statistically 
significant increase in the proportion employed (2.2 percentage points), as 
well as in the amount earned (an increase of $62) over the short term. 
(See table 2.) In the longer term, changes were small and not statistic- 
ally significant. (See Figure 2.) 

• For recipients, ESP produced welfare savings in the short term 
that, however, did not increase over the longer term. 

Short-term welfare savings for the full recipient sample were not 

sustained by the early sample followed over 18 months. 

Imoaots Among Other Su bgrouDa 

• Separate analysis in urban and rural areas showed gains in 
employment, as well as reductibhs in welfare, for urban 
experimentals. 

Employment impacts for urban applicants and recipients combined were 
statistically significant and larger than the gains registered for the full 
experimental sample. Impacts of urban applicants were especially large. 
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FIGURE 2 
VIRGINIA 



AFOE RECIPIENTS: .TRENDS IN QUARTERLY EMPLOYMENT 
RATES AND AVERAGE AFDC PAYMENTS 

(EARLY SAMPLE) 
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with emplbyment increasing in the short term by 7 percentage points over 
the control group level of 41.3 percent. Welfare savings were also 
stati.^tically significant except for the first quarter: in the year-long 
study period, urbar4 applicant experinentals received $101 less in welfare 
payments than urban applicant controls, for a reduction of 7.6 percent from 
the control level of $1,335. 

• ESP did not appear as eff^ In the raral areas, but scaail 
sample sizes made It difficult to assess results. 

In rural areas, there were no statistically significant impacts on any 

outcome measures and no clear trends. However, small sample sizes 

precluded precise analysis. 

• Different subgroups^ distinguished by charaQteristics sudh as 
prior workj welfare dependency and education^ benefited from 
ESP in different ways. Eard-tp-employ appiidant subgroups, 
particularly those who had had little prior work experience, 
tended to gain more than the more employable groups. 

Impacts on employment and earnings, as well as on average welfare 
payments, were concentrated in the applicant subgroups defined as 
"hard-to-employ:" i.e., those with less prior employment, longer welfare 
receipt, or less education. 

As one example, those without emplbyment during the year prior to 
program entry experienced an employment gain of 8.2 percentage points over 
the control group level of 36.5 pecent. The size of this gain was twice 
that of the gain for the group with recent employment. Welfare reductions 
for the group without prior work were $148 on a control group mean of 
$1,369. 

It is important to realize that while the impacts tend to be larger 
for the harder-to-employ subgroups, this does not mean that these groups 

o 
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attained high levels of employment. For example, the absolute level of 

employment was much higher for the group with prior employment (67.3 

percent) than for the group with no prior employment (36.5 percent). These 

fihdir^s do show, however, that programs working with those who would do 

very poorly on their own are capable of producing changes in behavior. 

• In cbhtrast to the applicants , bard-to-employ recipients did 
hot gain from the program^ except for recipients without a 
high school diplCHoa. 

Recipients without a high school diploma registered employment gains 

of i|.5 percentage points over f* control group base of 2^.5 percent. There 

were also statistically significant welfare savings of $211 over a control 

group average of $2,525. in contrast, there were no impacts for other 

hard-to-employ recipient subgroups. 

Benefirts-and-Costs x>f ESP 
The benefit-cost analysis reports on the overall gains and losses 
generated by ESP. This analysis differs from the impact analysis in 
several important ways. First, it estimates the program's effects nut only 
on earnings and welfare payments, but also on fringe benefits, tax 
payments. Medicaid, Food Stampsy Uhempibyment Insurance payments, and the 
administrative costs associated with these transfer programs. With the 
exception of Unemployment Insurance, these effects could not be directly 
measured and were therefore estimated using a variety of data sources. 

Second, using a number of assumptions, the analysis calculates effects 
and costs of the program that are likely to occur after data collection 
ended. This longer-range view is necessary, since most costs were incurred 
Quickly (i.e., when participants were still active In the program), while 
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benefits may accrue over a longer time period, as employed people continue 
to work, pay taxes and reduce their use of welfare. Estimates for this 
analysis thus cover a five-year period for each sample member, starting 
with the date of random assignment, and total results include both observed 
and extrapolated estimates. The extrapolated estimates are particalarly 
important to the overall results in Virginia because of the relatively 
short period of data collection, 

A third differenoe of the benefit-cost aiialysis is its concern with 
how gains and losses differ for different groups, often called 
'^perspectives. " The most important perspectives in this analysis are the 
welfare sample and the government budgets (Other groups examined in the 
full report but not discussed in this summary include society in general 
and everyone else in society except the welfare sample.) 

Two key questions are addressed in this benefit-cost analysis: 

• Does this study's vrelfare sample becoae economically better 
off as a result of ESP? 

• Are government expenditures Increased or decreased as a result 
of the program? 

Before considering the results, one must understand the assumptions 
made about future benefits. Because it is always uncertain whether program 
effects at the end of a study period will grow, stabilize or decline, 
alternative assumptions have been made about their continuation in this 
study of ESP. 

One assumption is that the magnitude of the iiLpacts observed during 
the two quarters immediately prior to the end of data collection will 
continue unchanged to the point five years after random assignment. This 
assumption is supported by the finding that overall employment and welfare 
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impacts for the early sample followed over the longer term were sustained 
or increased in the final fbllbw-up months. Moreover, similar trends have 
been observed in other studies of employment programs for welfare 
recipiehtSi 

An alternative assumption is that program impacts will decline after 
the study period by 22 percent per year, a rate observed in another study 
of a somewhat similar program and population. 

Thus, while the true long-term effects of the program are not certain, 
they are likely to lie within the range of the estimates derived using 
these two assumptions. 

# Overall ^ ESP benefited the individuals for whom it was 
operated. For both applicants and recipients, the economic 
gains generiated by the program outweighed the losses. 

From the point of view of this study sample, the main ESP benefits 
were increased earnings and fringe benefits from regular jobs, which were 
estimated to be $1*179 to $1,581 per experimental over five years, 
depending on the assumption about future effects. Another gain was a small 
increase in child care and other support service payments obtained through 
the program. These gains were partially offset by the higher taxes the 
individuals in this study paid ($11|9 to $201 per experimental) and their 
reduced transfer payments ($390 to $il99 per experimental), particularly 
from AFDC welfare and Medicaid. 

Overall, however, the benefits exceeded losses, as shown in Table 3. 
For the full sample, the net gain per experimental over five years ranged 
from $66^ to $905. The improvement was larger for applicants than 
recipients ($958 to $1,310 versus $i|85 to $663), because even though 
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TABLE 3 



ESTIMATED BENEFITS AND LOSSES OF THE EMPLOYMENT 
SERVICES PROGRAM PER EXPERIMENTAL, AFTER FIVE YEARS 



Wetfere Category 


- e b 
Welfare Sample Government Budget 


Put t Sempt e 

Total Benefits 
To tat LoBsea 

Net 6e1n or toss 


$1203 to $1605 $589 to $766 
-539 to -700 -430 

664 to 905 159 to 336 


Appt 1 ca n t s 

Totat Benefits 
Totat to.nses 

Net Gein or toss 


1629 to 2191 759 to 999 
-671 to -881 -253 

958 to 1310 506 to 746 


Re ci pi ent 8 

Total Benefits 
To ta I Losses 

Net Gai n or Loss 


978 to 1 296 51 9 to 667 
-493 to -633 -5^3 

485 to 663 -29 to 119 


SOURCE: Tebtes 5^5. 5^6« and 5.7 

NOTES: All estimates refer to everege expo r i men ta t -co n t r ot differences. Where 



eppropriete* e renge of estimates is preaented doe to the uncertainty of program effects 
after the study periods The first number in the renge essames that effects decline by 
22 percent per yeer; the second number essumes no increese or decrease; 

^For the welfare senplei benefits include increesed eernlngs, fringe 
benefits end ESP support services [e^g., child cere end transportation]; losses include 
increesed tex peyments end reduced transfers, primerity AFDC payments and Medicaid; 



For the government budget^ benefits Include Increesed tex payments redaced 
trensfersy end reduced trensfer progrero e dm 1 ni s t ra 1 1 ve costs. Losses include ESP 
opereting costs, support service expenditures, end the costs of education and training 
provided by community egencies. 
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applicants lost more in taxes arid transfer payments, their increased 

earnings arid fringe benefits were cchsiderabiy greater than their losses. 

• For the gdve^j:ment budget, the overall program ef 

positive. When directed toward applicants, the program 
produced a net budget savings within five years. When 
dlredted toward recipients, the benefits and costs were 
roughly even. 

From the government's pierspective, the main program benefits were 
increased tax payments, reduced welfare and other public assistance, and 
associated reductions in the administrative costs of these programs. 
However, public funds paid for the operating costs of ESP, as well as for 
support services and for most of the education and training services 
provided by community sgericies. The end result for the full sample was, 
however, positive, even when these costs were compared to the benefits of 
the program; gains exceeded losses by $159 to $336 per experimental. (See 
Table 3.) 

Most of this positive effect was due to the applicant sample, which 

produced a net budgetary savings of $506 to $7^*6 per experimental over five 

years, in contra^^t, among recipients, the estimates ranged from a net loss 

of $29 per experimental to a net gain of $119f depending on the assumptions 

about future effects. These results are probably best Interpreted to mean 

that, in providing services to recipients, the government breaks even on 

its investment in about five years. 

• The net costs of ESP were modest, although higher for 
recipients than for applicants. 

The average cost of operating ESP was estimated to be $388 per experi- 
mental (including participants as well as nbnparticipants) , ranging from 
$251 per applicant to $479 per recipient. The difference between appli- 
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cants and recipients reflects the fact that recipients were more likely to 
enter program services; applicants, on the other hand, left the program 
sooner. When expenditures for support services (e.g., child care and trans- 
portation) and for education and training provided by community agencies 
(above the costs of these services for controls) were considered, the aver^ 
age net cost of the program for each experimental increased to $^30, or to 

$253 per applicant and $548 per recipient. (Only a fraction of the sample 

_ - _ _ _ I 

used these services, which largely accounts for the low average cost.) ' 

It is important to recognize that these benefit-cost results do not 
take a number of othor important factors into account. First, it has been 
assumed that the higher employment rates of experimentals do not result in 
the displacement of other workers. Second, several intangible benefits and 
costs have not been measured, such as the benefits associated with 
society's preference for work over welfare. In addition, dollar values can- 
not easily be assigned to such social benefits and costs as welfare mothers 
spending more time working and less time with their children. These iimita^- 
tions should be kept in mind in interpreting the results of this analysis..: 
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CHAPTER 1 
INTRODUCTION 



This is the second and final report on the Employment Services Program 
(ESP), an initiative designed by the Commonwealth of Virginia to improve 
the employment prospects of individuals applying for or receiving; benefits 
under the Aid to Families with Dependent Children (AFDC) program J 
Virginia developed the program in response to the opportunity granted to 
states under the I98I federal Omnibus Budget Reconciliation Act (OBRA) to 
redesign the Work Incentive (WIN) Program, the major federally-funded 
employment and training system for the welfare population. ^ As authorized 
by OBRA, ESP has been operated as a WIN Demonstration Program, which 
permits variation in thei usual management practices of the program and the 
provision of WIN services. 

The OBRA legislation gave states the authority to explore ways of 
strengthening the linkages between welfare and work. One option, which ESP 
has used but varied slightly, is the Coiiimunity Work Experience Program 
(CWEP)^ a program that requires AFDiC recipients to work in unpaid public or 
private nonprofit positions; their work hours are calculated by dividing 
the welfare benefit by the minimum wage. Another OBRA provision, also used 
by ESP, allowed states ojperating WIN Demonstration Programs to centralize 
management authcirity in the Welfare agency* rather than to divide it 
between that agency and the state's employment service, as is the case in 
the regular WIN Program. 

ESP consists of a series of activities for AFDC applicants and reci- 
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pients classified WIN-mandatory . 3 The first activity is mandatory job 
search; Subsequent siervices can include CWEP> education and training. 

The Manpower Demonstration Research Corporation (MDRC) has been 
evaluating ESP under contract to Virginia's welfare agency* the Department 
of Social Services (DSS). The research design has three major studies — 
process, impact and benefit-cost examining, respectively, the implemen- 
tation of and enrollees' participation in the program; the effect of ESP 
services on enrollees' employment and welfare receipt; and the costs and 
benefits of service provision. 

Because the AFDC program in Virginia is state supervised but locally 
administered, responsibility for operating ESP has rested with the state's 
124 local welfare agencies. MDRC's evaluation covers ESP as operated in 11 
of these agencies, which roughly represent the statewide agencies. Ah 
interim report in 1985 described the early implementation and patterns of 
ESP participation. This final report updates that information and presents 
the impact and benefit-cost analyses. 

To provide a context for understanding the findings, this chapter 
first discusses the development of the program model at the state level 
before describing the settings of the 11 local programs, it concludes with 
a summary of the interim findings and the major issues addressed in this 
report. 

I. Program Baokgnourifi 

Virginia's WIN Demonstration began in January 1983i with single-agency 
management authority vested in the welfare agency, the Department of Social 
Services. Responsibility for planning the new program therefore fell to 
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state DSS staff, primarily members of the Adult and Family Service Bureau 
of the Division of Service Programs, the unit that had provided social 
services under the regular WIN Program. 

Although many of the employment programs currently being run by states 
are limited to a few communities and/or segments of the welfare population, 
Virginia opted for more inclusive coverage. The new WIN Demonstration 
Program replaced WIN iri all 12^4 local welfare agencies, and the Employment 
Services Program was thus operated statewide,^ targeting all WIN-mandatory 
individuals. This meant that the state's WIN resources, which had been 
sharply reduced jfoilowing the I98I cuts in the federal WIN budget, were 
spread over a large and very diverse caseload, which included people who 
had been WIN registrants for a long time. While this influenced the choice 
of services to be included in the state model, ESP was an unusual 
opportunity to examine a WIN Demonstration initiative targeted to a broad 
range of the welfare population. 

The model ultimately designed by the state intentionally provided only 
an overall framework for program operations. Virginia has a strong tradi- 
tion of local control and autonomy, dating from Colonial times, that had 
already formed the basis for the state's locally operated but state-super- 
vised AFDC system. Not surprisingly, when the state was given greater 
latitude to define WIN services, program goals and assignment policies, it 
decided to allow local agencies to shape ESP services to respond to 
conditions in their communities. (This allocation of authority was not 
possible in the regular WIN Program, where federally-determined goals and 
regulations had to be followed.) 

The state's interest in building flexibility into the program model 



was also based oh an interest in accommodating a wide range of opinions — 
both those of DSS staff and other important constituencies — about welfare 
dependency in general and the kinds of programs that should be offered to 
the welfare population to reduce dependency. To a certain extent, the 
state-planned multi-component ESP model reflected the diversity of view- 
points on these issues. Some influential Virginians advocated a mandatory 
program requiring registrants to participate in all activities; others 
believed that, as long as appropriate assistance was offered, welfare 
recipients could be relied on to voluntarily do what was necessary to 
begome self-sufficient. 

Opinion was, in addition, divided on the question of how well the 
regular WIN Program had served its enrollees. Some, especially in the 
state's employment service, believed that given the limited resources and 
the difficult problems of the welfare population, WIN had performed about 
as well as it could have. Others thought more could have been done, citing 
WIN'S record of working actively with only a limited proportion of the 
people registered. With the more demanding view of WIN predominating, 
particularly within DSS, an implicit goal of the new program was to 
increase activity levels beyond those in WIN. 

The ESP model that emerged from the planning process was a partially 
mandated sequence of services. DSS planners agreed to require job search 
of all women5 enrolled in ESP. In part because of their belief that 
welfare recipients have a responsibility to try tw find jdJs for them- 
selves, there was a consensus that this activity Would be useful for the 
full range of the caseload, as well as new AFDC applicants. 

Local agencies were also required to operate work experience programs 
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and to «5ffer opportunities" for education and training. 6 However, in 
contrast to job search, there was no agreement that all ehroilees should be 
required to participate in these activities. Work experience Was a new 
type of activity (one rarely used in the old WIN Program) and potentially 
controversial, and funding resources for education and training were 
minimal; consequently, for the latter, local agencir.s were advised to 
utilize existing community services provided by schools and other organiza- 
tions in their communities. Given these circumstances, planners recognized 
that it would be difficult to mandate participation. Thus, like all 
programs, the ultimate ESP mbuel represented a compromise, and its overall 
assumptions and intentions should guide interpretation of the results of 
this evaluation. 

Another important consideration was the nature of the welfare popula- 
tion, which is always partly influenced by the level of the state's welfare 
payments. Virginia ranks 36th from the top among all states in the size of 
its maximum benefit levels, with a family of one adult and two children 
receiving from $245 to $32? per month in 1984. in Virginia, payment 
standards also depend on the area in which the recipient lives (see 
Chapter 2 for further information on welfare payment standards in the 
state), although the state has established caps on monthly payments so 
that, regardless of area on f^ily size, benefits do not exceed from $372 
to $478 per month. These levels are hot very high, and employed or highly 
employable individuals in Virginia may not be either eligible for or 
attracted to welfare, lliis in turn has a bearing on the characteristics of 
the people enrolled in ESP. 
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il. The Program Morif^T 

Applicants and recijpients classified as mandatory were required to 
register with and jparticijpate in ESP in order to receive benefits. 7 Those 
not required were encouraged to volunteer for the program. As a condition 
of being approved for welfare, all eligible applicants had to first fulfill 
a special job search requirement — known in the program as "applicant job 
search" — which required them to provide proof of three contacts with 
prospective employers. After this, and if their applications for assist- 
ance were approved, they were then referred to an ESP Worker for assessment 
and possible assignment to program activities. 

These approved appliciantsS and all those who were already on the roils 
— including both existing WIN-mandatory recipients, as well as those who 
had just been determined WIN-mandatory were required to participate in 
either group or individual job search for up to four weeks. (This activity 
was called "recipient job search.") The content and requirements of this 
component were left to the discretion of local agencies, whose staff could 
offer one or both forms of the activity. Throughout their program tenure, 
the women were expected to take part in job search every six months, if 
they were still registered with the program, unemployed and not in another 
ESP activity. 

Reassessment occurred after each component, or at least every 90 days, 
except for persons employed part-time or enrolled in long-term education or 
training programs, who were to be reassessed every six months. Following 
recipient job search, the ESP Workers could send the enrollees back to job 
search or assign them to any of the post-jbb-search activities ~ work 
experience, education or training, which could be Adult Basic Education, a 
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general equivalency diploma (GED) program, vocational education courses 
provided either the public school system or a community college, or ah 
activity funded by the Job Training Partnership Act (JTPA). Other options 
were alsso available; 

Work experience in ESP combined elements of the CWEP approach with 
those in WIN Work Experience, a similar activity in the regular WIN 
Program. As in GWEP, ESP work hours were calculated dividing the grant 
by the minimum wage. As in WIN Work Experience, assignments were limited 
to 13 weeks (although registrants could be reassigned). Assignments were 
further restricted by state law to no more than 80 hours per month. 

During the early months of the evaluation, very low participation 
rates were reported in the post-job-seareh activities. To improve the 
possibility that these components could be fairly tested in the MDRC study, 
DSS established participation goals for ail of the ESP activities. These 
goals, later incorporated into letters of agreement between the state and 
the 11 agencies, were based in part on plans the agencies had submitted 
before program start-up. 

III. Program Settings 

The program settings Were varied. As shown on the map (Figure 1.1), 
one agency, Fairfax, is located in the northern region of the state; three 
— Newport News, Hampton and Chesapeake — are in the Tidewater area; and 
the remaining seven - Henry, Martinsville, Carroll, Grayson, Galax, 
Campbell and Pittsylvania — are in the southwestern and south central 
regions. DSS chose these agencies because they considered them generally 
representative of Virginians welfare agencies and because they met two 
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other critieria: their data systems Were appropriate, and they had ail 
expressed a strong interest in taking part in the study. 9 

The data in Table 1.1 establish two points. First, while there are 
striking differences in the areas served by the participating agencies, as 
a group they are fairly representative of the state as a whole. Second, 
although some agencies do not fit neatly into either the "urban" or "rural" 
category, the differences between the two groups are large enough to 
suggest that ESP posed different operating challenges and that program 
outcomes could differ along urban/rural lines. (Two of the areas classi- 
fied "rural" — . Galax and Martinsville — are actually small cities 
surrounded by large rural counties, and two classified "urban" are mixed — 
Fairfax County and Chesapeake.) 

To note one important difference, no public transportation was avail- 
able in the rural areas, where residents often lived far from the popula- 
tion centers. Also, as noted previously, the different benefit levels and 
labor markets in the urban and rural areas, as reflected in the average 
unemployment rates, could influence employment and welfare outcomes. 
Lastly> the urban and rural populations differed, as shown by their differ- 
ent levels of educational attainment, per capita income and types of 
occupation. All of these factors together suggest the importance of 
contextual factors in analyzing participation patterns and program effects. 

Table 1.2 shows that staff size was understandably larger in the urban 
agencies, with the Newport News staff largest of all. Staff composition 
also differed^ with the urban agencies simply expanding old WIN units to 
operate the new programs. Rural ESP Workers, on the other hand, were much 
more likely to lack previous WIN experience; they often had transferred 
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TABLE 1^2 



VIRGINIA 

ORGAN IZATIDNAL CHAR ACT ERISTICS OF THE EMFtOYMENT SEWI^S PRiBRAM, 

BY LOCAL ABBiCy 



Urban 

F«1 rfax 

Newport News 

Hsmpt on 

ChBsapeake 
Bare I 

Henry 

Groyeon 

GGtex 

Carrot t 

Hartlnavlt te 

PlttBylvaniB 

Ceopbet t 



NiKiber 

of 
Offices 



4 

3 



NuBbor of 
BipLoyjieht 
8a rv ices 
Workers* 



Nuiber of 
Enpldyneht 

Services 
Supervieore 



NuQber of Stiffing 

Levele Between 
_ _ESP Superyiior 
ind Agency Director 



19" 
5 
6 



1 

ShBp3d 
Worker 



1 

2 
1 



1 

3 

1 
1 



1 

Shered 
Unit 



2 
2 

1 

3 

0 
0 
0 
0 
0 

b 

b 



SOURCE: Biptoynent Service Program Statistical Reports , December 1963» and personal 
interviaws ^ndoct^ iri 1964. 

NOIES: •in eertaih agBnciesy ESP Wdrkors ere concurrently responsible for serving e 
portion of the Food Stamp caseload. 

b _ _ _ 

In Newport NewSi there is one ojnCrel office with 15 ESP Workers end two 

BBtellite offices with two ESP Workers eech. 
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from other divisions — the Eligibility Division, for example or from 
hon-WiN-related units of the Services division. In Galax, Grayson and 
Carroll (where ESP allocations did not allow even one full-time person), 
the shared Worker had a business, rather than a social services, back- 
ground. However, previous WIN experience did not mean that urban staffs 
were prepared to operate an employment program like ESP. In the past, 
these staffs had usually ^ust provided social services and were in- 
experienced in assigning and moving enroll ees through a number of cbmpb- 
nehts, as was required in ESP. 

IV. Ttie Eval uation Bpsign 

A. The Reafiareh Sfjnrt-j^.ci 

The following discussion and Table 1.3 summarize the questions, method- 
ology and data sources for the three parts of the ESP evaluation. 

The Process Analysis- To examine the program as operated by 
the 11 participating agencies, the process study describes the content of 
ESP services, analyzes patterns of participation, and discusses the 
implementation factors that may have influenced the program's effects. 

2; The Impact Analyst a* The impact analysis, the principal 
fociis of this report, addresses a number of questions about program 
effectiveness; Are there short- or longer-term impacts on the employment 
and earnings of enroll ees? Do the levels of welfare receipt or the size of 
the benefit cheeks decline? Do the impacts vary by different subgroups of 
the eligible population — i.e., between applicants and recipients, or 
between groups with more or fewer characteristics of welfare dependency? 

To answer these questions, the impact study uses an experimental 
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design. All eligible applicants and recipients were randomly assigned to 
one of three groups: 

• An experimental group eligible for ail ESP services; or 

i An experimental group eligible for only job search and work 
experience ; 1Q br 

i A control group not eligible for any ESP services. 
As seen in Chapter 2, random assignment ensured that experimental and 
control group members Were similar in all measurable characteristics except 
the receipt of ESP services. Thus^ any statistically sig;iificant differ-- 
ences in behavior after random assignment should result from differences in 
program treatment. Impacts were estimated by comparing the outcomes of the 
different groups. 

3. The Benefit-Cost Arna^y^s . This study assesses the net 
costs and benefits of ESP. In this analysis, net benefits will result 
primarily from increases in earnings and reductions in welfare and other 
transfer program payments benefits both observed in the study period and 
estimated by extrapolation — to cover a total period of five years. 
Benefits and costs will be analyzed from several points of view with the 
main emphasis on the results for this welfare sample and the government 
budget. 

B. The Reaearch Sampli:^ 

The interim ,*eport on ESP focused oh the 1,402 individuals randomly 
assigned to the program from August 1983 through February 1984. in 
contrast, this final report analyzes the full research sample — those 
randomly assigned from August 1983 through September 1984. Included are 
1,061 experimental s assigned to the Job Search/Work Experience group, 1,077 
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experimentais assigned to the All ESP Services group and 1*046 members of 
the control group. 

Follow-up data on participation, employment* earnings and welfare 
receipt for all sample members were collected through August 1985, per- 
mitting the program activity and outcomes of the latest sample members to 
be tracked for a minimum of nine months after random assignment. Those 
enrolled in the sample at the earliest point were followed for 23 months. 

V. I£ev Findi^s Ft'otti the Firat Rppnr^t 

The first report made it clear that, by allowing local agencies a fair 
degree of flexibility in operating ESP, the state presented them with a 
considerable challenge, one that was complicated by several factors. 
First, only a few months were available for planning before program start- 
ujp in January 1983f and consequently there was not enough time for agencies 
to institute innovative features. Second, resources were limited, 
particularly because of the federal WIN reductions nationwide in October 
1982. Third, ESP was a complex model to operate, especially for staff who, 
for the most part, lacked experience in running employment programs. The 
first report found that, faced with these kinds of constraints, the local 
agencies generally opted for simplicity and management efficiency rather 
than innovative programming. They also hesitatec* to impose requirements. 
While staff clearly regarded job search as mandatory and generally imple- 
mented* the activity that way, none of the agencies required participation 
in other components. 

The applicant job search requirement was also implemented as planned; 
most women who were asked to provide proof of three employer contacts did 




so, the applications of thosis who failed to meet the requirement were 
usually denied. The searches did not, however, generally lead to 
^ployment. 

The interim report also found that substantial numbers of women had 
also participated in recipient job search — close to half. Most of these 
particijpants (34 percent) took part in individual job search, while the 
remaining 11 percent were involved in a group activity, usually workshops 
offered in the urban agencies. 

Participation in pbst-job-search components, however, was low. The 
interim rate for work experience was only 6 percent, although this overall 
rate masked considerable variation, ranging from a low of 1 percent in the 
rural agencies to a high of 19 percent in Chesapeake. A primary constraint 
in the rural agencies was public transportation, but the most important 
factor in both rural and urban agencies, at least at the outset, was the 
attitudes of some staff toward unpaid work experience. No clear consensus 
had emerged about its value and, further, because staff did not want to 
alienate employers, the component was generally operated on a voluntary 
basis. Once participants reported that they liked working in the assigned 
jobs and supervisors registered satisfaction with their work, staff became 
more comfortable about assignment procedures and more confident about the 
component's usefulness for a wider range of people. 

Participation rates in training and education were also low but fairly 
consistent across agencies. Unlike work experience, these activities were 
always voluntary. According to staff, assignments depended primarily on 
the interest of enrollees, their qualifications and their motivation to 
participate. Although more access to training offered by JTPA-funded 



providers might have encouraged participation, staff usually did not have 
time to pursue the linkages heeded to set up referral channels. 

The interim report also described the content of ESP activities. 
Participants in group job search activities received substantial amounts of 
guidance and instruction on job-seeking and employability skills: i.e., 
appropriate work habits and attitudes. Assistance Was more limited in 
individual job search, in particular, staffs with large caseloads were not 
usually able to provide meaningful, individualized assistance when the 
women were first assigned. In the smaller conununities, where caseloads 
were small and knowledge of the women often extended beyond the formal 
caseworker/enrollee /relationship, ESP Workers were more likely to structure 
a woman's search and to follow up promptly when the end of the participa- 
tion period was in sight. 

A survey of 25 randomly selected participants in work experienc^e 
positions suggested that most liked their jobs and felt it was fair to t 
asked to work. While the jobs generally required few skills — ami 

supervisors reported that participants already possessed most of theiii 

both participants and their supervisors regarded their work as important 
and necessary to the sponsoring agency. Supervisors also judged the 
majority of women to be as productive as, or more productive than, regular 
new employees. In cases Where participants were initially judged deficient, 
in skills, they had, according to their supervisors, improved to the point 
of adequacy by the time of the survey interview. 

The interim report did not address the content of education and train- 
ing programs* Since these were operated by a wide variety of organizations 
and agencies within each comiiunity, it was not possible within the scope of 
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this research to docuinent the way in which these services were delivered i 
The important point about these two components is the availability of these 
services to the residents of a community who qualify for them. This meant 
that a member of any research group, including controls* could take part in 
these activities if they wished to enroll on their own. This issue is 
discussed in greater detail in Chapters 3, 4 and 5. 

VI. The Cnrr fi^ri^^pf^ 

Using a larger sample and a longer follow-up, this report updates the 
interim ESP participation patterns in all components and examines the 
program »s success in mandating job search participation. In addition, the 
report explores two new process issues: the participation patterns of 
important subgroups, with emphasis on applicants and recipients; and 
participation based on the proportion of people who remained eligible 
throughout the period studied. This second analysis more precisely depicts 
the program* s ability to reach the target population and judge px-ogratz 
accomplishments. 

The impact and benefit-cost analyses form the maj focus cf this: 
report. Short- and longer-term CTpioyment and welfare e;; J^ectn ar«^ esti- 
mates of benefits and costs over five years are presented i'o.: both ui/u full 
sample and ah early portion of the full sample, as well as -or appJi^i^nto 
and recipients and other subgroups based on degree of dlsati ^ntf*? 
will be discussed in Chapter 2, the fact that the existx^.s; WL'^-y - 
caseload is part of the sample makes this study one of partj ^ula'" . 
in MDRC^s Demonstration of State Work/Welfare initiatives. 

The report is organized as follows; Chapter 2 d:«easses the re& rch 
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design^ sample and data sources; Chapter 3 presents participation 
patterns^ while Chapter 4 describes the short- and longer-term impacts. 
Chapter 5 discusses findings on the benefits of the program relative to its 
costs* 



CHAPTER 2 

BESEARCH DESIGN. RR^RARCH SARPt.R ANEKDATA AflTTRrp ;,^ 



This chapter describes the research design, the research sample and 
the data sources used in the evaluation. The sample will receive parti- 
cular attention since the early .segment represents the WIN-mandatory 
cassioadl An Virgiida, which makes up about one- third of the total AFDC 
state v^a^eloao. Ao such, the sample includes a broader range of the 
velx^are population than is usually under study in the state employment 
prograuns in MDRC's Work/Welfare Demonstration. 

ESP was designed to involve most of the WIN-mandatdry caseload in 
program services, especially job search: WIN-mandatory applicants approved 
for welfare, WIN-mandatory recipients currently in the caseload, and 
recipients newly detemined WiN-mandatory (usually because their youngest 
'jhild had become school-age) .2 Individuals excluded from the research 
sample -- not only all of the men but also several groups of women are 
discussed in Section IV. 

All of the local agencies designated their existing caseloads for 
random :^ssignment as quickly as possible. Applicants and newly determined 
WIN-mandatory recipients were randomly assigned at a more or less even rate 
throughout the sample intake period. Thus, the majority of people entering 
the samjple between August 1983 and March 198^1 (over two-thirds) Were on- 
board recipients, including many who had been on welfare for some time. In 



contrasts most of the women enrolled between April and September 198^1 were 
either new applicants or recipients recently determined WlN-mandatory, fts 
a result, program impacts for the early research sample — i.e., those who 
entered the full sample first — will be of particular interest since this 

group contains such a large proportion of longer-term welfare recipients. ^ 

i 

XI. ^e Resfi;^r5h Design 

As explained in Chapter 1, the ESP model provided Job searchi work 
experience, and education and training services. The evaluation of this 
model used an experimental design in which members of the research sample 
were randomly assigned to one of two experimental groups — both of which 
were to receive a certain sequence of ESP services or to a control 
groups whose members could receive other services but were not eligible for 
ESP. However, enrollees assigned to any of the research groups includ- 
ing the control group — could enroll in education and training services 
Since these were not operated by ESP but were available on a community-wide 
basis. 

The inclusion of a control group in the research design was critical 
in assessing program effects since outcomes (i.e., job placements, 
departures from welfare) might not ail have been due to the program. Other 
research has shown that a significant proportion of AFDC recipients find 
jobs and leave welfare on their own in any given period. 3 Thus, what 
happened to the control group is an indication of what would have happened 
to enrollees without any ESP services.^ The difference in outcomes between 
the experimental groups (eligible jfor the program) and the control group 
(not eligible) will reveal the program effects. 5 
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The research plan called for gathering data on members of all research 
groups on a series of outcome measures: proportion employed, average 
earnings, proportion receiving welfare benefits and average welfare 
payments. Program impacts were calculated as a set of ordinary least 
squares regression equations. 6 x^e tables in this report indicate whether 
program effects on employment, earnings and welfare were statistically 
significant at the 1 , 5 or 10 percent level, with each level indicating how 
small the probability was that a given experimental-cohtrbi difference 
would have occurred by chance. 

The two experimental groups formed to examine the effects of differ- 
ent combinations of ESP services were: 

• The Job Search-Work Experience group, which received indivi- 
dual or group job search, followed sometimes by unpaid work 
experience in a government agency or not-for-profit organisa- 
tion. 

d The All ESP Services group, which ais-) roW<^r. ~vea individual or 
group job search. This could be followea tgr work experience 
a nd/or referral to an approved education or training program, 
including Adult Basic Education^ preparation for the GED, 
vocational education or JTPA-funded activities. 

Compariisdn of the outcomes of each of the two ESP service sequences with 
members of the control group was expected to reveal the effects of each 
combination separately. The validity of the rei^ults would depend on 
whether members of the two experimental groups received the different 
service components in sufficient numbers. 

As will be discussed in Chapter 3, participation levels in the 
different activities did hot in fact differ to the degree expected, with 
members of the Job Search-Work Experience group apparently enrolling in 
education and training services on their own. As a result, the levels of 



-23^ 

EKLC 



involvement of the two groupjs were too similar to justify separate analysis 
of the two sequences. (Services to controls are discussed iri Qiapters 4 
arid 5.) This final analysis thus combined data for members of the Job 
Search-Work Experience and the All ESP Services groups so that the outcomes 
of one pooled experimental group were compared to those of the entire 
control group. (Nevertheless, to ascertain if participation arid impacts 
differed by experimental group, the original analysis was carried cut. Its 
findings are briefly described when relevant in sections of Chapters 3, 4 
and 5.) 



iii. Thii Proofins of Entering- the R ea o arGh S^m p l^ 

Figure 2.1 traces the process through which the three research groups 
entered the sample. 

A. WIN^Manri a tory Ap pliGanta 

During the initial welfare eligibility interview, intake workers 
identified applicants who were WIN-mandatory and did not fall into the 
categories to be excluded from the research sample. (See Section iV. ) The 
workers explained that ESP registration was a condition of receiving 
welfare, described the program and the MDRC evaluation. Applicants then 
signed a statement which affirmed their understanding of the program and 
the requi 'oment. Only after this registration process wa? the applicant 
considercid an ESP enrollee.7 

Random assignment to one of the research groups occurred durirjg this 

initial interview. Staff members called MDRC for a cdmputer-gsnerated 

g . _ 

research code.*^ Intake workers informed those assigned to one of the two 
experimental groups that, as a condition of welfare approval, they had to 
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FIGURE 2.1 
VIRGINIA 

RESEARCH DESIGN FOR THE EVALUAtldN OF THE EMPLOYMENT SERVICES PROSflAH 
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first participate in an ESP activity known as "applicant job search": that 
is, they had to submit proof of contact with three employers during the 
period between application and welfare detcmination. (Potential employers 
had to Sign Applicant Job Search forms.) Appiicants did not and, in fact, 
could not participate in any of the regular ESP components until their 
applications were approved. 

in Virginia, applications for AFDC welfare must be acted on by the 
local agenc:v?s within 45 days. After the decision, the files of the 
approved applicants assigned to one of the two experimental croups were 
transferred to an Employmeht Service Worker on ESP staff, who contacted the 
enroiiee for service assessment. 

ReciDientg Detf^rm-i ned WIN-ManHa-f^y 
Recipients whose status was about to change from WIN-voluhtary to 
WIN-mandatory were interviewed first by ah eligibility officer in what is 
known as a redetermination interview and, then, as in the case of all other 
ESP enroiiees, by an Employment Services Worker for service assessment. 
(Newly determined WIN-mandatory recipientr could be randomly assigned 
during either interview, with the eligibility officer making the decision.) 
During the redetermination interview, recipients were registered with ESP, 
signed a consent form and received an explanation of the program. 

Members of the experimental groups learned more about the ESP 
compononts in their assessment interviews with the ESP Workers. Those 
assigned to individual job search were given forms on which to record their 
employer contacts, while those going to gi^oup job search were told the date 
of their first class. Experimentals not assigned to these activities were 
placed in a pending category. 
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e. Qn^Boarri Reni pi prit,-^ 

bn-board WIN-mandatory recipients eiigib: for the sample included 
women who had not as yiet been registered and assessed by an ESP Worker, as 
well as those who had been registered and assessed between January and 
ikugust 1983 (some of whom had already ;). . ticipated in individual or group 
job search since the prdgram^s incepts. . in January). Women in the first 
--oup were randomly assigned durxn^ their assessment interviews, while 
women in the second group went to a reassessment interview for random 
assignment. As in the case of other experimentals, those assigned to an 
experimental group were giv- . either individual joj searcl^ employ^.- contact 
forms, scheduled for group job search, or placed in the ;.:iding category. 

D- ^he Snhedple of fir^ndom As5lgnTTiP>n1-: 

Random assignment was scheduled to begin in August 1983 and continue 
through September 1984. (See Table 2.1 . ) While the process proceeded 
smoothly, the agency start-up dates differed, as did their timing in 
assigning their current WlN-mandatory caseloads. Most agencies began in 
September but two of the urban agencies, Fairfax and Chesapeake, Waited 
until October. And, while ail of the agencies pha?sd m the caseload in a 
relatively short period of time, Newport News assigned nearly MO percent of 
its caseload in the first two months. 9 In fact, almost 90 percent of all 
enrollees vho entered the research sample in these two months had been 
randomly assigned at one of ti?2 three Newport News offices. 

The pace of random assignment in the other offices accelerated between 
October and December. Thirty-five percent of ail enrollees (1,118) were 
assigned during these months* with five of the rural agencies, along with 
Newport News, completing this process for a majority of their individuals 
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DISTRIBaTlON OF THE RESEARCH SAMPLE, BY MONTH OF 
RANDOM ASSIGNMENT AN p_i EL FARE STATUS 
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57.9 


107 


Ba.l 


207 


72.3 


Apri L 


99 


65. J 


101 


87 „4 


200 


78.6 


May 


92 


7 2. / 


04 


91.8 


176 


84.1 


June 


85 


79.3 


47 


94.3 


132 


88.3 


July 


93 


86.6 




36 .7 


138 


82.6 


August 


102 


94. a 




F3.4 


154 


66 .8 


Se pt an be r 


71 


10 0. 0 


3 I 


lOU. J 


102 


100.0 


Total 




12B7 


1897 






3184 

















SOURCE: TebuLations from MDRC CMent InformBtion Sheets. 
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by the end of the year. By January 1984, 1,495 or 4? percent of the 
research sample had been randomly assigned. 

The process slowed down from then on. By the end of March, an 
additional 807 people (or 25 percent) entered the sample. At this point, 
most of the ejcisting caseload had been assigned. From April through the 
end of random assignment Lii September, mo^t people entering the sample 

(another 28 percent) vere applicants and newly determined WiN-mandatory 
recipients. 

IV • The— Researoh— Samp^ p 

The research sample formed by random assignment contained 1,287 

applicants and 1,897 recipients, a total of 3*184 people or about 54 

percent of the WlN-mandatbry caseload in the 11 urb?n and rural agencies 

included in the ESP demonstration. 

As noted previously, some portions of the welfare caseload were 
excluded from the sample during a screaning process in the assessment 
interview. Males were excluded because their labor market behavior was 
expected to differ significantly from that of wcraen. (Their numbers were 
also so small that it seemed unlikely that this group couid be analyzed 
separfa. tely . ) Certain groups of women were excluded because they were 
judged unlikely to participate in an ESP component other than job search: 
i.e., those who had previously taken part in ESP work experience, education 
or training programs,' those who were .already enrolled in full-time educa- 
tion and training programs; and WIN-mandatory recipients with children 
younger than six. 1^ Volunteers were excluded partly because their 
demographic characteristics, background and motivation were expected to 



differ irbm those of the rest of the sample members. In addition^ agency 
staff did not want to deny them ESP services through possible random 
assignment to the control group. 

Table 2.2 shows that in total, about 46 percent of aii those screened 
for entry into ESP were excluded from the research sample and placed into a 
non-research group during interviews by ESP Workers. Table A.I describes 
the reasons for their exclusion. ^ ^ Nee: ly 60 percent were parents of 
children under six years of age; another 14 percent were already in 
full-time education or training programs. About 9 percent were volunteers. 
The demographic profile in Table A. 2 shows that the non-research group 
tended to be younger and better educated than the research sample, and that 
on average its members had lower earnings during the year prior to random 
assignment. 

A. CharaQt^ndrafei^la^ of th e Ftri 

Random assignment succeeded in generating three research groups Whose 
miembers v.ere similar in background characteristics and life experiences. ^ 
(See Table A. 3.) By definition* they were single female heads of house- 
hold, with blacks predominating in all groups. As shown in Table 2.3, the 
average sample member was in her early thirties and had two children. I^pi- 
cally, she faced significant barriers to employment, having not completed 
high school or worked during the previous year. She also had been on 
welfare for more than two years. (Table 2.3 also describes the demographic 
characteristics of the sample by agency.) 
B. Subgroup Samplfta 

Research findings based oh the full sample may mask significant varia- 
tion. Separate analysis of important subgroups helps to determine whether 
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8AM^E »RQLLfe«T BY RSEARCH STATUS^ 
tOC^ A6»CY| AND PEhIOD OF RANDON AS8IBNHENT 
tAUGUST 18B3-SEPrEC3ER 1664 SAMFLE) 





1 


Raaaarch Saaple 




1 ■ - 
NorrReaaarch SaapLe 




Job 












Agahcy and Psriod 
of Randott Asvlgiwant 


6aarc^ 

Work 
Exparlahca 


Alt EBP 
Barvlcea 


Controls 


Total 


Not 
Randomly 
Aaal gned 


Parcaht Not 
AandcMly 

Assigned 
















August 1883 - Mftrch 1884 
April - SaptSDber 1864 


142 


146 
84 


142 
80 


420 


260 


50.4 


— 

Total 


224 


230 


222 


676 


600 


47 iO 


Naw&f^t-Nan* 














August 1863 - Harch 1864 
April - August 1864 


228 
65 


234 
60 


230 
56 


882 

1 DO 


ool 
146 


44.7 


To tel. 


283 


284 


268 


875 


789 


47.7 
















August 1863 - Itorch 1864 
April SeotaDber 16B4 


113 


116 
46 


113 

44 


344 
133 


OCD 

156 


54.0 


Total 


156 


164 


157 


477 


482 


50.3 


ana w\m 














August 1863 • Harch 1864 
April * Sapteaber 1864 


116 
45 


119 
41 


114 
44 


351 
130 


357 
204 


50^4 
61.1 


Total 


163 


160 


166 


461 


561 


53.8 


fluraV*eafte4^fi 














Auguat 1863 - »ferch 1864 
April - Saptanber 1864 


161 
64 


164 

65 


160 
61 


465 
180 


211 
78 


30.3 
28.4 


Total 


225 


228 


221 


675 


280 


30.1 


AU Aoanclaip 














August 1863 - March 1864 
April - Saptanber 1664 


762 
288 


781 
286 


759 
287 


2302 
662 


1885 
837 


45.2 
48.7 


Total 


1061 


1077 


1046 


3164 


2732 


46.2 



Tabulatlone frooi HDflC Client Infomatlon SheetE and randos aeElgnment telephone 
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8ELECtEp_CHARACtiR26T|C8„0F TM£ RESEARCH BAMPtE 
AT THE.TIME.OF 8AKD0M A88IBNMENT, BY tOCAL AGENCY 
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certain categories of people benefited more or less from the prbcram than 
others. This section discusses the principal subgroups analyzed and the 
reasons for selecting these groups. 

1- Appilcanta — and — ReGipifintia- Ihroughout the following 
chapters, findings will be reported for the full research sample and for 
applicants and recipients separately. The reasons for a separate focus oh 
these subgroups are threefold. First, applicants and recipients entered 
the research samjple in different proportions at different times throughout 
a year-long study period so that, depending on the time period examined, 
their measures of participation or outcomes could differ. For instance, a 
sample enrolled in the early months of the program was analyzed to deter- 
mine longer-term program impacts, and this sample was largely composed of 
recipients from the existing caseload. Their behavior was quite different 
from that of the later sample. 

Second, prior research suggests that the labor market behavior of 
applicants and recipients differs, partly because of their different educa- 
tional backgrounds, prior work history and the extent of their welfare 
dependency. As seen in Table 2.4, over a quarter of the applicants had 
never been on welfare, and those who had received it previously had stayed 
on the rolls a shorter time than recipients. The proportion of applicants 
who had held a job during the year prior to random assignment (57-2 
percent) was more than twice as large as that of recipients (22.9 percent) ^ 
and applicant earnings during this period ($2,291.22) far exceeded the 
recipient average ($406;i»1). Applicants also were more likely than 
recipients to have received a high school degree or its equivalent (50.9 



-34- 

78 



TABLi 2.4 



VIRGINIA 

SELECTED CH AH A CT Efl 1ST I CS OF THE RESEARCH SAMPLE 
AT THE tiME OF RANDOM ASSIONMENT, BY WELFARE STATUS 
UUBUST 1883 - SEPTEMBER 1884 SAMPLE] 



Pi^B ra cterl etl c 


AppL i ca ht e 


Ra ci pi e n t 6 


Local Agancy l%] 
Fal pf ex 
Neip or t 
Hai pt on 
Chaaa paa ke 
Rural Agencies 


27.4 
27.7 
14^6 
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21 .4 


17. O*** 
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33^0 


34.b^*« 
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Hi spe hi c 
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41 .7 

55.0 
1 .J 
0.2 
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70.1 
1 iO 
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- c 

Total Ssnple 
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1887 
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NOTES: Oi Btri buti on B nay not add exactly to 100.0 percent because of 



roundi ng. 



[ conti nued ) 



-35- 



79 



ERIC 



TABLE 2»A [cdntl nued] 



t-ength of AFDC de pendency lis self- reported, Beceuse of 
Inaccurite snsierG or rtscor d1 hg error 6» percehtagee displayed here only 
approximate actoat length of prior AFDC receipt, 

jj 

Calcolatad froe U nan ploy line nt Iheurahce earnings records from the 
CoRBonii eel th of Virginia. 

c 

For satected cha racte r 1 sti C6» eaMple slzee nay very up to 9 

sfisplc points due to Dleelng date* 
d 

Chl-sqoere teet Inappropriate due to low expected cell 

f raq aenci e a. 



Differences between welfare etetuees 
significant using a two-tailed t^test or chl-aquara 
levelss ♦ = 10 percent; = 5 percent; = 1 per 



areetetistlcelly 
teet at the following 
cent. 
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percent versos 38.8 percent). Many applicants thus possessed at least 
minimuni job market credentials, and this spell of welfare might have been 
only temporary support. ^3 

Third, the different enrollment procedures for applicants and recipi- 
ents might have influenced participation levels. Applicants went through 
two extra steps before their assessment and assignment to an ESP ccmponeht: 
completion of applicant job search and grant approval. At each point, 
particularly the second, some applicants left the program, about 32 percent 
because their grants had not been approved. In contrast* most recipients 
were randomly assigned during their assessment interviews ~ a point much 
closer to ESP piurticipatibn. 

2. iiariler and Later Enrol lf>e>a: Erirollees Were also subdivided 
into two groups depending on when they entered the research sample. The 
first group was the early s^pie — those enrolled from August 1983 through 
the end of March 1984; the second, those randomly assigned from April to 
September 1984. (See Tables A. 4 and A. 5.) The earlier group of enrollees 
represented 72 percent of the full research sample — 82 percent of all 
recipients and 58 percent of ail applicants. (See Table 2.1.) in contrast, 
applicants made up the majority of the group that entered after April 1, 
1984. The early sample, with its large proportion of on- recipients, 
not only represents the WiN-mandatorjr caseload, but also is an important 
source of infonnatioh on longer-term impacts. Follow-up for this group was 
six quarters, or 18 months, after random assigiment in contrast to four 
quarters, or one year* for the full sample. 

Analysis of the separate seumples* in addition, allowed the examination 
of any Impact differences that could have been caused by a change in 
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program operations or the labor market^ both of which coald affect the type 
of enrollees entering the research sample. For example, in August 1983, 
Virginia's uhempldyment rate was 5.2 percent and in September 198^1, it Was 
4.4 percent. (See Table A. 6. 3 The passage of new welfare regulations or 
changes in how local agencies screened the welfare caseload also might have 
infiufcrnced participation and butcdmos. 

3 • Yar ia t i pn By Lf>fiP "-^fe^e^pt^ Differences be tween 1 ocal 
agencies, some related to urban and rural location, might have resulted in 
different Impacts. Virginia has a strong tradition of iocai autonomy, 
which is reflected in the relatively high degree ol operational discretion 
given the separate agencies. Staffs could adopt different ccSministrative 
procedures and vary the content of services withia broad guidelines, is 
well as the requirements of each. Tlie quality of the services could also 
differ^ in addition to staff attitudes and qualif ications* and political 
and economic climate . (See Chapter 3 for a discussion of progv^am 
implementation. ) 

As noted in Chapter 1, the 11 agencies in the demonstration included 
large cities, small towns and rural areas. Table 2.3 shows that the 
majority of enrollees in rural areas were white? while the four urban areas 
enrolled primarily blacks (although racial compositibh did vary consider- 
ably in the urban agencies). Higher proportions of urban than rural 
enrollees had held a job during the year prior to ^ndom assignment and had 
graduated from hi^ school or received a GED (although, once again, signi- 
ficant variation characterized the urban agencies). Since individuals from 
urban areas constituted nearly four-fifths of the research sample, sample 
characteristics as a whole largely reflect their experiences. 
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The demand for labor in a region may influence the ability of welfar 
recipients to find jobs. Table A. 6 demonstrates that uneitiployment rates, 
which provide a strong indicator of the demand for labor, differed consider- 
ably among the 11 areas studied. Average un^ployment rates ranged from a 
low of ?..7 percent in Fairfax County to a high of 9^3 percent in Grayson 
County during the two years covered in this report. 

in Virginia, welfare regulations determining the maximum amount of 
welfare grants accordiA. to family size varied by region. Payment 
standards were based on calculations of the cost of basic necessities in 
local areas, reflecting an effort to equalize the living standards of 
welfare recipients throughout the state. Thus, differences in the 
availability and amount of welfare payments could influence decisions about 
whether to apply for -eir -e, remain on Welfare or - geek employment. 

Table 2.5 show. :e three levels of payme pr=indards within the 
state. AS of July 1984, a family of three in Fairfax and Ha .-on counties 
(Group iii) could receive a ms- imum of $327 per month, compared to $259 in 
counties in Group and $245 in Group I. The vai^iation among payment 
standards was greater for smaller than for larger families. 

4. Additional ^^t>grQUPS> one of the purposes of evaluating an 
employment program targeted to hard-to-«npldy persons is to detemine 
whether the program did in fact benefit those who faced serious barriers to 
employment. Chapter 4 will address this issue by examining program impacts 
for subgroups who differed in characteristics that arr usually used to 
define level of employabiiity : prior work experience, extent of previous 
welfare dependency, educational level and family size. 
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TABLE 2.5 



VIRG INIA 

VAHIATiONS OF DOLLAR AMOUNTS FOR PAYMiNTS STANOARDS, 
BY FAMILY SIZE, AGENCY AND YEAR 



PAYMENT STANDARD 



Fam 1 L y Size 


Group I 


b 

Group 11 


- 

Group III 




July 1 SBO July 1 984 


July 1980 Juty 1 984 


July 1980 Juiy 1984 


One r.'ireon 

Two 

Three 

Four f 

1 

MaxInuiD j 


115 121 

1Q1 131 

223 245 

PB3 29B 

3c3 372 j 


138 145 

203 214 
255 269 
305 ^21 

381 402 


193 203 
258 272 
310 327 
360 379 

453 478 



Catcutatlons frora the Commo nw b a L th of Virginia Grant Etigibitity 
and Procedures Manuat- 

NOTES: Payment S ta n da r u i s d ef i he d as the amount of benefits a family of e 

certain 8l2B w ou t d r a ce 1 v e 1 f the family has ho aerned or unearned income and 
there are no required edjuetaente reflecting prior underpayments, overpayments, or 
pro-retlone l--r exampter If a faioily wee rox eU§ibLe for the entire month j . 
a 

_ ^^^^^'P ^ con6if;ts of Campbell, Carroll, Galax, Grayson, Henry, and 
Pi 1 1 6 y I V b n i r. . 

b^ - 
broup II CDneiBt.^ of CheBapeake» Ha r ti nev i I I e, and Nawport News. 

c — _ _ 

Group III conelets of Fairfax and Hampton. 
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The three analyses in this report — process, impact and benef i t-coiit 
used somewhat different research samples and follow-up periods. The 
process sample included 3,18^1 people ~ the full reaear-h sample who 
were followed for nine months. An early part of this sample was tracked 
for six additional months, for a total of 15 months of data. 

The impact sa^iple in Chapter ^ consisted of 3,182 people: i.e. ^ the 
full research sample except for two controls who did not have Social 
Secj-i.y numbers. This sample was followed for 12 months after random 
assi^wrj^^nt. kr, ■^dditiohai six months of data wei'e available for the early 
por r Uie full sample* as discussed previously, for a total of 18 
months in foliow-up to determi- lohger-tsm trends. 

The benefit-cost study (Chapter 53 is also based on the full sample^ 
but makes use of :ai available fdllow-up. For example, in the case of 
earnings data, the earliest enrollees have data for seven quar\ .rs 
subsequent to the quarter of random assigrment, while the latest enrollees 
have only three follow-up quarters. 

Because of the variety of data sources needed to estimate program 
costs, certain calculations in the benefit-cost analysis were performed on 
randomly selected subsamples from the group of 2*810 experimentuls and 
controls who entered the research sample between August 1983 and June 1934. 
Subsample sizes ranged from 111 individuals for determihing the cost jf the 
work experience cr^ponent to .,7r>8 individuals for calculating the cost of 
support services. A total of 2,446 experlmentals and controls were 
involved in at least one subsample. All data gathered from the suDsamples 
were weighted according to the distribution of sxperimentals and controls, 
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as Well as to the distribu xon of applicants and recipients, within each 
agency. 

analyses of par* - eipa'ibn, welfare and employment outcomes^ as 
Well as benefits and costs* drew oh a variety of data sources. 

CAierit Inf<>rmrition ghe^tLs— r^mi , designed by MDRC. constituted the 
primary source of demographic data about the research sample. Applicants 
answered the CIS questions during eligibility interviews by intake workers, 
and recipients also did so during assessments with ESP staff. 

"rheH^.rgiDta Client Information Sys tem iVACTS) maintained by ail but 
one of the 11 agencies provided data used to .rack program activities and 
monthly AFLO grants!^ until the end of data collection in September 198{' 
VACiS was modified for the evaluation to include data on both assignment ^ 
program activities and participation. However, since ES? staf. tended to 
report assignments more aocurately than participation^ the decision was 
made to use assignment data to track enrol lees' involvement in activities. 
Wfc - aii people assigned to an activity actually participate^ quality 

c checks revealed that assignment rates were somewhat under- reported 

in VACiS and therefore approximated quite closely the participation rates 
reported in individual case files. However, because Newport News had very 
different job club assigdme;:t and participation rates, state staff checked 
the case files of those as;iigned to the dubs to look for actual 
participatibh. 

All local agencies except Fairfax Covrx^/ ^rvrcr-.-ra program tracking data 
and AFDC Program records into the VACiS system. Fairfax County submitted 




AFDe data from its own ^stem to MDRC.15 por tracking data in this agency, 
MDRe coiiected program records for a random subsampie of mCTbers in the two 
experimental groups (183 of the experimentais) and entered them into 

the computerized analysis file used for this report, 

The Commonwealth of Vlrglhia Un^mpjHayt^j^ h Insitr^^ .Svstftm contains 
dati on earnings and UI benefits. 17 ^DRe collected data on earnings from a 
year prior to random assignment until the end of June 1985. Records of UI 
^nefits coYored the period from January 1982 through August 1985. 

For all of the sources described above, Table 2.6 shows the length of 
foliow-up data available for participation^ i.FDC payments, earnings and Oi 
benefits according to the period of random assignment. Other sources of 
data, however, were necessary as well, particularly in the benefit- cost 
analvsis. To calculate costs, MDRC consulted fiscal records from local 
af^^nciesf enrollment records from schools and community colleges, the 
Vl-;.nia JTPA inforaation system, and s^:atc 2. on the administrative 
costs o. transfer programs suc;h as ..u^eaid. Food Stamps, AFDC and 
Unemployment Insurance. 

MDRC also conaucted interviews (between March 1984 and February 19853 
with a random subsampie of 47 participants in work experience and their 47 
supervisors. The survey instrument was the same one used in other states 
in MDRC»s Dononstration of "tate Work/Welfare Initiatives Where the 
programs included structured work experience, interviews with participants 
lasted about 45 minutes and covered tneir understanding of the mandatory 
charactsr of the program* their level of satisf actio:: with the assignments, 
their assessment of the usefulness of the experience, and their judgment of 
Hie equity of the work-f or-benef its approach. Hour-long interviews with 
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VIRGINIA 

LENGTH QF AVAILABLE FOLLOW-OP, BY 
RANDOM ASSIGNMENT PERIOO ANO DATA SOURCE 





I 

Hon th 6 


'^v js r ta re ) of 


Av ei t ab t e 


Date On: 




Period of 

Random Asaignment 


AFOC 

-.a 

Pey men t 6 


Pr og r am ^ 

Tracking 


Qua r t e r 6 

— c 

ta r ni nge 


UI Banef i te" 


Sam pt e 

Size 


Aug UBt- September 1983 


ii;^ - 26 


21 - 22 


e 


24 - 25 


377 


October- December 1983 


22 - 24 


18-20 


7 


21 - 23 


1118 


J anufl ry- ^e rch 1984 


19 - 21 


15 - 17 


b 


18 - 20 


BQ7 


April-June 1984 


16-18 


12 - 14 


5 


15 - 1? 


508 


July-SBptembe r 1964 


13 - 15 


9-11 


4 


12 - 14 




HonthB (Quarters} og 
Avelleble FoLLow-Up 


12 


9 


3 


1 2 





NDT';:^ For UI Benefits and AFDC Peyment6» month 1 ie defined as the 

ncnth In «h1ct. random aGeighment occurB. 

'^Fbr Prhgrem Trackl hgy mc^ith 1 beglni on the dey of rendom 
eBBlgt.ment; month 2 beglne 31 deye Later. 

c 

Ear n1n§B data Is referred to in calendar quartere; quarter 1 »s 
the quarter In which rehdbm eBslghmeht be cure. 

d 

Month B of ev a i I abl e follow-up refere to the maximum amount of 
folLow-up date avallabl.^ for ell enrolleee. 
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supervisors provided information about productivity of l> - c^; ..oipau^.Sj 
the importance of the assignment to the operation of the K :7'kj='. ;.>•=> as well 
as their assessment of the Employment Services Program and i ^ :,^aff. 

Qualitative research sources also inf oraed the analysis. MDRC 
conviucted interviews with state and local agency staffs, examined program 
documents, and observed implementation of the program during site visits. 
For information about the background of ESP, MDRC interviewed state 
officials and representatives of agencies and organizations that contri- 
buted to the development of ESP and also examined operating guidelines and 
legislative documents. 

vii. Aaalyais- Jaaitea 

Clarification of the different ways in which AFDC payments and Ui 
earnings are recorded and the consequences of these differences for 
quarterly outcome measures will facilitate understanding of the analysis of 
program impacts in Chapter ^. As stated previously, AFDC payments are 
reported monthly^, but earnings are reported by calendar quarters (for 
example^ January thrcigh March). ihe month of random assignment could 
therefore be ar^r one within the three-month period, with the quarter of 
random assignment sometisias including up to two months of data prior to 
sample entry. Thus^ the qt^arter of random assignment throughout this 
report is not a true follow-up quarter for mployment and earnings impacts. 
In contrast, since welfare data are reported monthly and aggregated into 
three-month time periods beginning with the month of random assignment, the 
first follow-up quarter is a true impact quarter for welfare outcomes* As 
a resuxt of these reporting "differences, quarterly data for welfare and 
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emiployiD^nt outcomes do hot usually cover the same tiine periods. 

Another issue is that, in ,ome instances, the use of the UI data 
system underestimates total earnings because of unreported earnings and 
reporting lags* UI earnings records do not include of f-th e-books earnings, 
or earnings for people who have moved or who work out of the state. In 
addition, eL^,i.ioyers may hot report earnings for some workers, particularly 
domestic workers. Despite these issues, thcsre should not be any major 
biases since experimentals and controls should have been affected to the 
samk3 extent. 

Lastly, because of the typical lags in employers reporting wages to 
the 01 system, data for the full sample were available through only four 
quarters of follow-up, although a follow-up of six quarters was possible 
for the earliest sample members* 



CHAPTER 3 

PA jrg^ PART-IC-I PA-TKN 



1; Introduction 

this chapter discusses the nature of actl : in Virginia's 
ISpioyxnent Services Program and the extent to whi ' ogram enrollees 
participated in them. As noted previouoly, DSS had expected that most of 
those eligible for the program would participate in the mandated job search 
component; in fact, the model called for job search participation hot only 
initially but also on an ongoing basis if enrollees were still unemployed 
and registered with the program. 

In contrast, the state model did not mandate participation levels in 
work experience, education or training, leaving that to this discretion of 
the local agencies. DSS planners, nonetheless, anticipated substantial 
participation in all ESP components, envisioning the program as one in 
which enrollees would take part in a variety of employment-related 
activities, with job search functioning as the touchstone as enrollees 
completed other components. 

According to the design, enrollees were to start with job search and, 
if still uneirployed after its comp.etibh, progress to a second activity — 
Work experience, tr-^inihg or education. ^ Fbi lowing the second a;^tivity, 
'mt^mployed enrollees were to be assessed ag^ + n and reassigned to the saiie 
component or another one. At minimum, enrollees wer to remain accive by 
'returning to a job search ?'>tivity every six rnon^-'^s. Over time, the> .ould 
c^uaiably build up nev gk:^lis to market to employers as a result of 
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participating in one or more post- job-search services. 

The in'iii im report indicated that these broad objectives formulated at 
the state ?! were accepted by the local agencies and were at least 
partiai::> met during the early months of the study. That report found 
substantial participatioh in job search activities, particularly individual 
job search. Participatioh in other ESP components was more limited, 
although the rates, particularly for work experience, varied widely by 
agency. 

This final analysis differs from the •:nterim one in several ways. 
First, the interim report exGimined the participation patterns of enro3 1ees 
who were randomly assigned to two experimental groups (see Chapter 2) 
during the early months of the study, from August I983 through the end of 
February 1984. In contrast^ this analysis lool's at the program involvement 
of the en' • re sample^ those randomly assigned to the experimehtai groups 
<r^:^ August if'83 through September 198^1. 

Second^ the full sample has a longer foilbw-up than the earlier one. 
In the interim report, only six months of follow-up data were a-^^stiable. 
In this analysis^ the full sample was tracked for nine months after random 
assignment, and a large share 73 percent — was followed for 15 months. 
This longer follow-up is important given the r'llti-componeht nature of the 
ESP design and the duration of some of its ac i.vities, particularly 
education and training. 

This analysis is also more comprehensive. It not only updates the 
interim findings using the full sample, it looks separately at the 
participation patterns of applicants and recipients^ as well as oi;her 
subgroups. it also seeks to arswer new questions. After describing 
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cumulative participation patterns, the chapter analyzes participation by 
the ninth month after random assignment, taking welfare and employment 
status into account. This new analysis reveals that many experlmehtals 
could not be served by staff because they had left welfare or the progr^ 
shortly after entering it either because of emplojntDent or for other 
reasons. Many of the nonparticipants had not been subject to the program's 
participation requireient for very long. 

The chapter is divided Into several sections. Section II ana^/zes the 
cumulative participation patterns and also presents the mor^- detailed 
assessment of the program's success in reaching its targets' ^aseload^ 
discussed above. The next section describes the activitie in which 
enrollees took r--^! Section iV looks at the participatidn jf important 
subgroups within thz target population, while the fifth examines participa- 
tion patterns over v^'^e. 

II. Cufflu^lativ e Partidlp atriof^ Rates 

A. ^a^lcipafc^len-fete s for the Full .Sample 

The questions add; . st-^ed in this sect-'cn are the most basic: 

i What proportion of the full sample of enrollees 
participated In ESP? 

o How did participation patterns vary by component? 

• Wiii.t proportion of applicants and recipients ever 
participated in ESP and Its components? 

An ESP enrdiiee's experience with the program began in an in-depth 
assessment interview with ah ESP Worker.^ Applicants were only assigned to 
an ESP Worker for assessment after their applications for welfare were 
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approved. As Chapter 1 explained, these Individuals had to first take part 
in an Applicant Job Search^ which required them to show proof of three 
contacts with potential employers as a condition of grant approval. 

The purpose of the assessment interview was to determine the 
enrdllee^s strengths and weaknesses and what^ if any, barriers she might 
have to participation and, hence, employment. Test r ^w.^ally were not 
administered during this interview. 3 The ES kor -^s^d his or her own 
judgment to decide whether the enrollee should be Immediately assigned to a 
job search activity or to an Inactive status. 

Two points are Important in the following discussion of overall 
participation rates. First, the overall volume of program activity was 
larger than that suggested by observation of the experimental sample. 
Members of the non-research group (see Chapter 2) were eligible for 
services, and soae were assigned to components. 

Second, overall rates mask the variation in participation rates among 
the 11 agencies in the study. In general, the variation was greatest in 
the work experience component, where several factors influenced the level 
of worksite assignrrents. In the rural areas, for example, the availability 
of transportation and the remote residences of many recipients were f^^^^f^*- ? 
to be considered before making every assignment. Tr.e two most i: 
reasons for agency variation, howevst% we^e ones cor-non to all age. or * 
staff attitudes toward the value of work experience as part of an employ- 
ability development strategy, and the degree of emphasis on implementing a 
cohesive assignment policy. Appendix D presents more information on agency 
performance and the factors that influenced patterns of local ESP activity. 
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Table 3.1 presents the participation patterns of the two experimental 
groups. (See also Appendix Table CI for data on performance indicators by 
V rearch group and local agency;) in this analysis, assignment rates serve 
as a proxy for participation, which is therefore defined as "being iassigned 
to a given activity." As seen in the table, 80 percent of enrolXfccS were 
initially assessed or reassessed in interviews with an ESP Worker within 
nine months of random assignment. The majority — 58 percent — were 
assigned to an ES? component within this period. There were no significant 
d .Terences between the two experimen'.al groups in the degree to which they 
were assessed or participated in ESP* 

As intended in the model, enrollees first participated in job search, 
t;he only mandatory component for all enrollees. Somewhat nore than half 
participated — MO percent in individual job search and 15 percent in a 
group activity.^ Again, there were no differences between the rates for 
the two research groups. 

Enrollees were less active in thr^ post- job-search activities, as 
indicated earlier. Appendix Table C.2 shows that together, only 20 percent 
of all enrollees entered work experience^ education or training, either as 
the only activity after random assignment or followiiig participation in job 
search. Component by component ^ Table 3.1 shows that approximately 10 
percent participated in work 'experience, and a similar proportion, about 12 
percent, participated in education or trainirig; almost twice as many were 
assigned to education as to training. (On average, those assigned to work 
experience participated for 19^ hours,) The table also reveals that when 
the two research groups are compared^ the research design did, to some 
extent, affect the allocation of services. Members of the All ESP Services 
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VIRGINIA 



Att AFDC: NINE-MONTH PE R FO RMAN CE IN DI CATO FOR EXPER I MENTAL 

BY RESEARCH GROUP 
lAUGOST 1983 - Sl?TEHBER 1984 SAMPLE] 



T r 





JbbSearch- 


ALL ESP 




PerfoPHance Indicator 


Nor k Exps r1 ehci 


Se rv 1 ce 8 


ToteL 


a 

Aaea 888 d 




80 .6 


78.9 


79.7 


Ever Ac t V e 




59.2 


57.3 


5B.3 


Pertlclpated In Job Search 
Indlvlduat Job Seerch 
Group Job Seerch 

Acti V 1 t1 86 


51 .8 
40.7 

14.1 


50.3 
40 .1 

15.3 


51 .0 
40.4 

14.7 


Parti cl pe ted 1 n 
Expe r 1 ence 


Work 


13.5 


5.5 


9.6*** 


Pa r t1 cl pa tad In 
or Educa t1 on 

T ral n1 ng 

Educa t1 on 


T ral n i ng 


9.1 

3.5 
5.7 


14.1 

5 .n 

9.1 


11 .6*** 
4,3* 
7.4*** 


b 

Oe repi stared 

* __ c 

Due to Sanctioning 


38.8 
3.8 


45. B 
3.9 


42. 3*** 

3.8 


d 

iPLaceaent Rate 




1 3 . 8 


14.4 


14.1 


Total Number of 
Expe r 1 Be n ta L B 




1061 


1077 


2136 











SOURCE: Catcutatlone from the Virginia Automated CLIent Infbrnetlbh Syetem 
and from program activity data cottected from ce8e file recorde of e random 
eubeampLe of e xpe r 1 ne n t eL b In Felrfax County^ 

NOTES: Performence Indlcatora are defined ee ever assigned to a perticuLjar 
component or etatue. 



( con t i nue u ] 
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TABLE 3i1 (contifiueaj 



Performance indicatorB a r a ca t c □ t a t a d as a percentage of tha tbteL 
perione 1 h the 1 ndl ca tad r eeea rch _ group In att egehclee but Fairfax. In Fairfax 
Cbuntyi performance indicators a r e ca t cu t a t ed ae a percentage "of a random 
aubeanpLe of 1B3 experlBentets whoee case fUee were reviewed. The reeulta from 
Fairfax County are freighted to equat the total hunber of Fairfax e xpe r i ma n t a L b 
In the re see rch eanple^ , 

e^ _ _ - _ - J 

AsBbjfimant is defined for appMcants and redatarminad WIN-sehda tdry 

reciplBhte bb contact with an Em pi oyma n t Sa rv 1 ce b Worker after random 

asslgnmBnt. For thoaa raclpientB wh-o wara WiN-manda ti ry baforfl tha start of tha 

researchi assaBBBipnt la daf 1 nad aa raaBaaaamant at the point of random 

bbbI gnmant. 

b-. ___ _ 

DsrBjgi Btrati oh 1e dBfihBd bb baing daregiBtarad according to ESP 

program recbrdB or failing to raceiva AFDC paymanta at any tima during tha nine 

months following random iaEBignment, 

c_ 

Sflhctioning ratea are definad as referral for eanctions. 

^^Prbgram placement information Ib basad on empLoymenit that Is raported 
to program staff. Program placament data will not ba usad to measDre Impacts. 

A chi-squarB taat was applied to dlffBrsncas between reeearch groups. 
StB ti Btl cal b1 gni f i cBncB IbvbIb are indicated as: ♦ = IB percent; = 5 
parcant; ^ 1 parcant. 
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group were more often assigned to the full array of ESP services, while 
those in the Job Search/Work Experience group were more than twice as 
iikeiy as the Ail ESP Service?* group to hold a work position. 

The fact that more of the All ESP Services than the Job Search/Work 
Experience group participated in education and training percent 
versus 9.1 percent) should not obscure the magnitude of the Job Search/Work 
Experience sample enrolled in these programs. Some may have been referred 
to these services by ESP staff, but most probably took part on their own. 

This last factor highlights an important point about the ESP model. 
While state planners had: specified that educational and training "oppor- 
tunities" be available to enrbllees, funding constraints dictated that 
local programs rely on existing programs elsewhere in the community, which 
were available to all residents who qualified for them. Hence, many of 
those assigned not only to the Job Search/Work Experience group but also to 
the control group took advantage of these services on their own, as is 
discussed in Chapter 1|. Although control participation had serious 
research implications, as explained in Chapter i|, the participation of 
members of all three research groups iir these activities should not be 
construed as an implementation fault; the model had, in fact, encouraged 
agencies to use these services in making referrals. However, the 
similarity of the experimental-control rates suggests that ESP, on the 
whole, was unable to significantly increase the receipt of such services by 
experimentals beyond that which they would have found on their own. 

Overall, U2 percert of all enrbllees were deregistered from the pro- 
gram by the ninth month after random assignment; the rate was significantly 
higher for the Ail ESP Services group. Only a small proportion percent) 



-BU- 
SS 



were referred for sanctioning because they were not in compliance with the 
program's requirements. 5 There were no differences between the two 
research groups on this measure. 

Placement rates, as presented in the tables in this chapter, are 
frequently used by employment programs to measure success. However^ these 
rates reflect only employment reported to program staff and recorded in the 
DSS client information system. The ratei presented in Chapter based on 
Unemployment insurance records, are a more reliable source of emplbymeht 
data^ 

An assessment of these performance measures led to an important 
decision in the ESP evaluation. It was clear from the service 'receipt of 
the two experimental groups that the treatment of the two groups was quite 
similar and that any differences were not sufficient to justify analyzing 
the treatment groups separately. Data for the two experimental groups were 
thus subsequently pooled. The rest of the participation analysis, as well 
as the impact and benefit-cost analyses, will use data from the two 
experimental groups combined. 

B. Par^ticlp ation Patterns of Appiicants and Recipients 
Chapter 2 indicated that applicants and recipients had different 
demographic characteristics, as well as different backgrounds in prior 
employment and welfare receipt. Table 3»2 shows that the two groups also 
had very different participation patterns. In general, far fewer appli- 
cants participated in the ESP components than recipients, but significantly 
more applicants were deregistered from, or left, the program (54.8 percent 
compared to 34.0 percent). 

By program activity, 60.2 percent of recipients took part in some job 
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VIRGINIA 

ALL AFDC: NINE-MONTH PERFDflMANCE INDICATORS FOR EXPER I MENTALS, 

BY iELFARE STATUS 
[AUGUST 1883 - SEPTEMBER 1884 SAMPtE] 



fdrfdrinance Indicator 




Reel pi ente 


T 0 1 e I 


Aese 666 d° 




BS.B 


81.0 


79. 7*** 


Ever Active 




41 .7 


69. 4 


58. 3*** 


Pe r 1 1 c1 pa te d in 


Job Seerch 


37.2 


6 0.2 


51 .O*** 


Individual Job Search 


29.9 


47 .4 


40. 4*** 


Group Job Seerch 








A c i; 1 V 1 t i e E 




8.B 


iB.d 


14. 7*** 


Pa r t1 c1 pa te d In 


Work 








Experience 




3,2 


13.6 


8.5*** 


Pa rtl c1 pa t ed In 


T r e 1 n i h g 








or Educetldh 




8.8 


13.5 


11 .6*** 


T re 1 n 1 n g 




3.1 


5.0 


4.3«« 


Educe t1 oh 




5.7 


8.6 


7.4** 


Deregi stared 




54.8 


34.0 


42. 3*** 


Due to Senctldhlhg 


3.4 


4.1 


3.g 


Ptecement Re£e^ 




B.B 


17.7 


14.1 


lotet Nomber of 
ExperlnehteLGl 


857 


1261 


2138 



SODRCEi CelcuLatlbhB from the Vlrglhle Autoaeted Client Informetlon Syetein 
and frdin program activity dete coLlectad from case file record e of e rendora 
eubeempLe of experlmehteLe In Feirfex County. 



NOTES: Performance irdicatorB are defined ae ev^r eeeigned to e perticuLer 
conponent or stetue. 

(conti nued) 
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TABLE 3.2 ( cdhti nued] 



Pfi»'*'o»'"ence IndlcBtcrs are celculeted ee e percentege of the totet 
pepBopB in the Indicetedreeearch group In ell agenciee but Felrfex* In Felrfex 
County t pepformence^jndl cators are catcuteted ee e percentege of a rendom 
eubcemple of 1 B3 a xpe r 1 ne n t el a vhoee ceee fUis nere reviewed. The reeutte from 
Fairfax County ere weighted to eqaet the totel number of Felrfex e xp e r 1 m e nt a t a 
in the reeeerch sample. 



_ Exper imenta t B are comprlBed of both Job Seerch^Work Experience end 

All ESP ServiceB reaeerch groupsi 

a ^ _ _ L : : 

ABeeaamdnt 1 B defined f or app I 1 ce nt b end redetermined WIN-nandatory 
raclpiehts ea contact with en Employment Services WDrker efter random 

assignment. For those reel pi entB If ho were WIN^mehdetbry before the stert of the 
rasaerch, assessment 1e defined es reeasessment et the point of rendom 
a as i ghme h t . 

: . * 1 : - 

Ueregi str eti on Ie defined as being dereglstered eccording to ESP 
prbgrem records or felling to receive AFDE peyments et any time during the nine 
months following random esslgnment* 

c„_ _ : 

Senctloning retee ere defined es referret for eahctlone. 

^Progrem pi e cem e n t i nf orm e t j on Is beeed on emplbyBient thet is 
reported to progrem steff. Progrem ptecement data will nbt be used to measure 
1 n p e c t s . 

A chi-squere test was appMed to differences between weLfere 
stetuaes. Stetisticel slgnificence levels are indiceted as; ♦=10 percent; 
= 5 percent; »»» = 1 percent. 
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search activity compared to 37^2 percent of appiicantsi (Aimbst twice the 
prbpdrtibh of recipients participated in both individual and group job 
search compared to applicants.) In work experience, recipients predominat- 
ed, with a 13.6 percent level compared to just 3»£ percent of appiicantsi 
The pattern was less prdnouhced in education and training, where I3.5 
percent of recipients participated compared to 8.8 percent of the 
applicants. 

These participation patterns, as well as the higher applicant 
deregistratibns, are explained in part by program regulations specifying 
that applicants could not participate in ESP services until their requests 
for aid had been approved. In fact, as discussed in Chapter 2, almost bhe- 
third of the applicant experimental s had not received any welfare payments 
within a year after random assignment. In this analysis, these individuals 
were recorded as having been "deregistered^ " However, Appendix Tabl<i C.3 
indicates that, even when participation rates were calculated on a base 
excludlhg those who were not assessed — e.g. , applicants whose grants were 
not approved wore not referred for assessment — applicants still had lower 
participation rates than recipients. 

C. Partictba tion and Continuing Eligibility 

Thus far in this chapter, participation has been defined as the per- 
centage of enrollees whb were ever assigned to a given activity; that is, 
those who "ever participated" during nine months. Several problems are 
inherent in this definition. For one, this laeasurement makes no distinc- 
tion between thbse whb piarticipated for a day and those who participated 
for a much longer peribd, and therefbre gives no indication of the duration 
or intensity of participation: 
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A second problem is that these rates may tanderstate the program's 
achievements. "Ever participated" rates take a static view of participa- 
tion, suggesting that all people reEaained eligible throughout the study 
peribdi but that many somehow avoided program participation. Actually^ a 
fairly large number of enrol lees only remained in the program for a short 
period of time^ leaving welfare or being deregistered from the program for 
a number of reasons (i.e.^ remarriage, the birth of a child, iore than 30 
hours of employment). Staff often had no opportunity to work with these 
enrbliees during their short program tenure. Other enrollees were employed 
part-time and were still registered with the program because their low 
incomes continued to qualify them for supplementary benefits. These 
individuals were often judged to have satisfied program requirements 
because they were working what was considered to be an appropriate number 
of hours. 

When these findings (or reasons for hdnparticipatioh) are taken into 
consideration, the extent of a program's ability to serve the targeted 
caseload is more evident. Figure 3.1 shows that by the ninth month after 
random assignment, only 6.5 percent of the applicants and 11. percent of 
the recipients had remained enrolled throughout the study period, but had 
failed to participate and did not have jobs. This is the group that the 
program failed to reach. to state it another way, 93,5 percent of the 
applicants and 88.6 percent of the recipients had satisfied the program's 
requirement, either by participating or finding employment, or had been 
deregistered because they had left the program or were no longer eligible 
for welfare, (See Appendix Table C,^ for additional data on enrollees' 
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FIGURE 3.1 
VIRGINIA 



AFDC APPLIQANTS_AND RECIPIENTS: PRGGRAM. 
EMPLOYMENT. AND WELFARE STATUS IN THE NINTh 
MONTH AFTER RANDOM ABBIGNMENT 
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participation^ employment and welfare status at the ninth month after 
random assignment.) 

It is not surprising that more "continuously-enrolled" recipients had 
failed to participate in job search. As seen in Chapter 2, the recipient 
group exhibited greater prior welfare dependency; staff may have found them 
harder to work with. While 711 percent of all applicants had also 
previously received welfare, on the whole they were more likely than the 
recipients to have had recent employment. The important point is that, 
among both applicants and recipients^ the proportions of both groups who 
had not participated or worked were very small, indicating that ESP had 
succeeded in implementing the Job search participation requirement. 

It should be noted that^ in addition to the initial job search 
requirement, enrdllees were expected to return to that activity as long as 
they were unemployed and still registered with the program. Unfortunately, 
the data do not exist to determine if this ongoing job search requirement 
was in fact satisfied, as was the initial one, discussed above. Long-term 
foiibw-up data cover only a portion of the necessary sample, and some of 
the sample members were not continuously subject to this requir^ent 
because they had left the program or welfare. However, the available data 
show that at least 25 percent of the experimental sample participated two 
or more times in job search within an 18-morith fbllow-up period-^ so it is 
clear that a reasonable level of ongoing participation in job search was 
taking place. 
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111. Descripti on of ESP Services 
A. Job Search 

As indicated in Chapter 1, the Ciodel proposed by the state gave local 
agencies discretion, within broad guidelines, to determine some of the 
content and requirements of all components, including job search* Appendix 
Table C.5 shows the variety of job search activities that the agencies 
offered, as well as the different requirCTents. Group activities, as well 
as individual search, were offered in some form in ail of the urban 
agencies and, for a period of one year, on a rotating basis in three rural 
agencies. 

Generally^ group job search services were provided in the form of a 
two-week workshop, 7 in which participants were helped to identify their 
assets and skills^ to understand the basic qualities that employers seek in 
employees, and to develop appropriate employment goals. Under the guidance 
of staff and with the support of other participants — workshop 

enrol lees learned how to approach employers and to set up interviews, and 
how to present themselves positively during these appointments. Telephone 
techniques were taught in all of the workshops, but during the period 
covered by the evaluation, telephones were not available, or their use was 
limited, in some agencies. Not all participants therefore could call 
employers under staff supervision, a commbh feature of job search 
workshops. 

Individual job search, in contrast, was hot as structured.^ Enrol lees 
were required to make a specific number of personal contacts with potential 
employers and record the results on a form provided by the agency. ESP 
staff prepared enrollees for their search during the assessment interview 
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in which they received their assignments. In most instances, there was ho 
further contact with the enrollees until the reassessment interview after 
the search had been completed; Given the limited time of the interviews, 
many aspects of job-hunting could not be discussed ^ and staff success in 
guiding enrollees depended to a large degree on their previous knowledge of 
the person, in some rural agencies, this was possible and proved helpful, 
but, as a rule, enrollees in this component received far less assistance 
than those participating in group services. 
B. Vork ExDerience 

This ESP component was structured in part to conform to the OBRA CTEP 
regulations; work hours were calculated by dividing the welfare grant by 
the minimum wage. In contrast to what OBRA allowed, however, assignments 
to public or private nonprofit agencies were not ongoing, but limited to 13 
weeks^ although the state model specified that enrollees could be 
reassigned to the activity. In fact, 23.7 percent of those assigned to 
work experience (66 people out of 278) were assigned more than once. 

In Virginia, as in five other states in the MDRC Work/Welfare 
Demonstration offering structured work experience, a survey was conducted 
with a random sample of work experience participants and their supervisors. 
The total number in Virginia was i|7 of each. While a preliminary analysis 
of the first 25 worksite interviews was presented in the interim report, 
this report examines responses from the total sample; In general, the 
findings^ summarized below, are similar to those found for the partial 
sample. 

First, most participants (38 of the ^7 interviewed) believed that the 
work requirement was fair; more than half said they felt better about 
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receiving welfare now that they were working for it. Second^ the jobs* for 
many reasons, were not seen as "make^work." Almost all participants liked 
them, with most believing that they had learned something by doing their 
worki Almost 80 percent of both the participants and supervisors said that 
their work was necessary to the agency, and all expressed the opinion that 
the work was Important; 

Lastly, while the worksite Jobs in Virginia required few skills and 
supervisors indicated that most participants already possessed the ones 
needed for their jobs, in almost all instances where participants lacked 
them, the skills had subsequently been acquired. Appendix E describes the 
worksite survey and findings In greater detail. 

C. Education and Tral tttng 

With one exception, local agencies did hot have sufficient funds to 
operate traihihg or education programs themselves. 9 Generally, as 
recbmmehded in the program design, they referred enrollees who needed and 
were interested in such activities to community resources. These Included 
cbmmuhity colleges, the public school system, training programs run by JTPA 
service providers, and privately operated training facilities. interviews 
with staff suggest that Adult Basic Education classes and short-term skills 
training prbgrams offering clerical and hea]th-care Instruction were 
popular with enrollees. 

Problems did arise, however, in bbtaihihg sufficient access to these 
services, especially bnes that interested ehrbllees. Initially^ the state 
had intended that local agencies would ievelbp new referral relatibhships 
with the appropriate schools, institutibris and JTPA-fuhded prbgrams. It 

-eat- 



ing 



was hoped that a wide choice of activities would encourage ESP enrollees to 
participate. 

ESP staff in all agencies had little difficulty in making arrangements 
for Adult Basic Education classes, GED preparation courses and vocational 
r^Qhbbls and community colleges, ^ey had established working relationships 
with these service pr'oviders and had come to understand the entry 
requirements. They also knew the specific people to whom enrollees should 
be referred for help in registering for classes and, if necessary, 
obtaining financial assistance for tuition payments* 

For a number of reasons, working relationships were slower to develop 
with personnel in the JTPA Service Delivery Areas (SDA) and the service 
providers themselves, although staff in all agencies reported having made 
progress in setting them up. Two agencies, for instance, had come to ah 
agreement with the JTPA SDA oh a tracking form for ESP referrals. However, 
staff generally indicated that much more remained to be accomplished, 
primarily in the area of timely commuhica tioh. Information was needed oh 
such issues as what courses were offered^ how referrals were to be made, 
and how enrollees* progress should be monitored after referral. While JTPA 
service providers frequently expected ESP Workers to be invdlvf^d in 
attendance issues for their enrollees, ESP staff indicated that they often 
did not receive data about attendance quickly enough. Enrollees sometimes 
established patterns that resulted in termination before the ESP Worker was 
aware that a problem existed. 

It is worth noting that JTPA replaced the Cbmprehehsive Employment and 
Training Act (CETA) system as the major publicly-fUnded employment and 
training program for the disadvantaged at about the same time as ESP 
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replaced the WIN Prbgram in Virginiai Therefore, both ESP and JTPA staff 
were facing pressure and difficulties in implementing their new programs. 
This fact^ as well as the time it usually takes for new relationships to 
develop, largely account for reports from ESP staff that fully satisfactory 
working relationships were not always in place during the period covered by 
this evaluation. 



IV. Participa t ion^^^^ rhs for Subgreuea 

Thus far, the analysis of participation has focused on the full sample 
and the principal subsamples of applicants and recipients. This section 
looks at the patterns for important subgroups within both the full sample 
and the applicant and recipient subgroups to see if some segments of the 
targeted population participated at higher rates than others. Analysis of 
these data, together with qualitative datai from staff interviews, suggest 
how registrants were assigned to J^SP cbmpbherits, 
A. Participation 141- Urban arid Rural Ar e as 

Data on the urban arid rural areas presented in Chapter 1 laid the 
foundation for the discussion iri this chapter of the possible influence of 
environmental factors in shaping the delivery of program services, parti- 
cularly in the roi'al areas. Several such factors might affect participa- 
tion levels. First, as noted previously, public trarispbrtatibn was nbn- 
existent in the rural areas (and sometimes limited iri the urban regions) 
and the population was widely dispersed. Second, drily some bf the rural 
agencies were able to offer group job search services, and then brily Tor 
part of the period covered by the study. Third, community resources for 
education and training were in more limited supply in the rural areas than 
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urban areas, and Worksites were hot always In locations that enrollees 
could conveniently reachi 

Table 3*3 reveals that, as expected, applicants arid recipients In 
these different envirorSents exhibited different participation patterns, 
although not always the ones anticipated, the environmental problems noted 
above J as well as the analysis of enrollees » characteristics In Chapter 2 
showing clear urban/rural differences in education and past emplojrment 
had suggested that urban participation levels might be higher than rural 
ones. In contrast. Table 3.3 points out that rural women were often more 
acflve than their urban counterpartsi 

Both applicants and recipients participated In individual "job search 
at slghlflcahtly higher rates in the rural areas, but participation In 
group job search was understandably much lower in the rural areas, where 
group services did hot always exist. The higher overall rural rates in job 
search activities (61 percent) compared to urban areas (48 percent) is the 
most notable and sbmeWhat surprising finding. 

Work experience rates exhibited a different trend, although the 
applicant rates were not as much different as low in both urban and rural 
areas: 3.7 and 1.6^ respectively. However, recipients in urban areas — 
where public trahsportatibh was usually available — were more than twice 
as likely to be assigned to work experience as rural recipients. 

In contrast, despite fewer opportuhl tles^ rural applicants were twice 
as likely as urban applicants to have participated in training (5.3 versus 
2.5 percent), ari they were just as likely to have taken part in an 
education program (6.^1 versus 5.5 percent^ respectively). As was the case 
for the applicants, more recipients in the rural versus urban areas took 
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part In training activities, again despite the limited supply arid 
transportation constraints. As discussed in Appendix D, staff iri three of 
the rural rgencies vere particularly successful in obtaining training 
assignments for their enrol lees; In one instance, the ESP Worker had a 
longstanding working relationship with the local JTPA service provider; in 
another, the Worker was a momber of the Private Industry Council. It 
should be noted, however, that such relationships were rare, and that 
forging new linkages with JTPA training providers took considerable time — 
time staff needed to implement other components for which they bore direct 
responsibility. 

The recipient pattern was quite different for education, where, in 
contrast to applicants, recipients in the urban areas were almost twice as 
likely to have participated in educational programs than rural recipients 
(9.5 versus 5.3 percent). Several factors in the urban environments may 
have influenced these levels — enroiiees there appeared to want to attend 
school, particularly community colleges; Adult Basic Education, GED and 
commuhit:^ college courses were available in a great many locations; and 
enroiiees (at least according to staff perceptions) seemed to realize the 
need for a better educatidh in a:^ urban economy; 

B. PafM:ieiB a^i^^t-berns for Other Subgroups 

Tables 3.4 and 3.5 present participation data for other important 
subgroups within the applicant and recipient categories. The general 
pattern that emerges, at least for job search, showed that the potentially 
more diff icult-to-emplby among both groups ~ those with the largest 
families, older wcmen^ those with greater prior welfare dependency and with 
little recent employment were active in ESP to the same or to a 



mv, 3.4 



VIRGINIA 

AFDC APPUaHil NII1&-I0N1II PERFIIRIU iNDlCATOflS 
r-Ofl SaECTEO SUIGROUFS of EWEeifENmS 
(AUGUST 1983 - SEPTEfBER 19B4 SAHUE) 



J 

si 

0 



FerfomancB indicator 


Niber of Cfilldran 


filgh School 
01 plane or GEO 


Enpioyed in year 

Prior to Rendom 
e 

Aesignmont 


AFOC Hietorj 


b 

f 


Age 


Ona 


T»o 


Three 
or Hore 


No Yes 


^lo tee 


Never 


2 Yaare Hore Than 
or teas 2 Years 


than 3D 


Sd np 

Older 


c 

Aseesead 


— 
63,5 


— 
60.4 


— 
68.6 


67.4 


58.4*** 


5D.6 


64.1 


52.3 


59.9 


70.7*»* 


80.7 


63.8 


Ever Active 




40.2 


44.4 


44.5 


39.0 


42.3 


41,2 


34,4 


44.4 


43.8* 


42.0 


41,2 


PartlclntBd in Job Search 


3d .3 


3di4 


41.9 






3/. 4 


37.2 


28.8 


39,4 


4(),4** 


35.9 


37.8 


Indivlduat dob Search 


29,5 


29.7 


33.8 


32.8 


27.^» 


26.0 


31.1 


23.8 


33.7 


30,5* 


26.4 


30.6 


Group dob Seerch Actlyitles 


10.5 


9.0 


9.3 


10.0 


9.5 


11.7 


8.5 


6.5 


8,5 


12.5* 


10.4 


9.3 


Pflrtjcipitfid III Work 


























Eprlence 


3.5 


4.1 


1.2 


2.4 


4.D 


2.9 


3.4 


3.9 


2.1 


3.7 


2.9 


3.4 


Pertlcjpited in Training 


























OP Iducstlon 

VI whUWU III VII 


11.0 


7,6 


6.2 


1D.0 


7.7 


7.8 


9.5 


8.0 


8.2 


9.6 


11.9 


7.r*» 


Training 


4.5 


1.7 


2.S 


3.8 


2.5 


2.9 


3.3 


2.0 


1.4 


5.0»* 


5.5 


1.8»'* 


Education 


6.5 


5,9 


3.7 


6.2 


5.2 


4.9 


6.2 


6.0 


6.8 


4.8 


6.4 


5.3 


--d 

Oeregi stared 

a 

Due to Senctlonlng 


50,3 


58.0 


60.3** 


53.0 


56.6 


55.5 


54.1 


66.5 


54.0 


49,2*»* 


51.7 


56.6 


2.5 


5.1 


3,0 


1.6 


5.1«»» 


3.1 


3.6 


3.4 


3.5 


3.4 


2.9 


3.6 


Ptacement Rate 


7.S 


9.4 


10.6 


6.9 


10.5* 


6.2 


9.1 


6.5 


7.9 


10.7 


9.1 


9.4 
























Total f>jiiber of Apptlcanta 


397 


289 


161 


416 439 


343 


514 


199 


280 


376 


308 


547 







ERIC 



[continuedj 



T«LE3.4 (continuedj 



_■ . SOURffi: RlculaiioiiB RffiC Ciient infowatioii SheBte* the Virilnla Autonefod Client irifbrnetlon Systm end f rora 
activity dsti coUicted m m file records of a rahdbn siiSisBinple of experiiiientets in Fairfax Gouhty. 



NOTES: Paifbriance indicators are definad as aver assignad to a particular cofiponent or statue. 

^ rtrfijraance Indicators m calculated as e of tiia total persons in ttia indicated reuirch group in all 

agencies bot Fairfax. In Fairfax County, perfomancB Indicatci- are calculated as a percentage of a rahdoii subsaiple of W 
BxparlMntBle «hon case files Rere reviied, The results from Fairfax County ere Rsiehted to equal the totcl niber of hirfax 
experiiientals in the research soiipie. 

Experiwntals are comprised of both Job SearcFtork Experience and Alt ESP Services •eseardi groups, 

'ftrsons were considered empli^ed during the year prior to random asEignnent if thay had UI earnings for any of the 
foor prior perters, 

b 

CBlculBtlons from NDRC Client Infonnstion Sheets, 



ces 



li_ _ _ _ _ ___ 

Aansieht is defined for applicants and redeteimined t(.?l-»andatory recipients as wntact »lth an Eijployinent Servi-w- 
Worker after randtl aBBipent^ ftr thba recipients who «ere Mandatory before the start of m research, assseient is defined 
as raassesenent at tha point of random assignment. 

d 

Deragistration is defined as baingjeregistered according to ESP progran records or falling to receive AFDC payments 
at any tiifl during the nine months folining random assi jmsnt. 

Sanctioning rates era defined as roforral for sanctions, 

Progra placenent information is based on emptoyment that is reported to progran staff, Progri placiant data wili 
not be used to measure impects. 



^^^^ test was applied to differsncBS be^sen subgroups. Statistical significance levels are indicated as- * = 
10 pepcenti = 5 percent; *»♦ = 1 farcent. 
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significantly grisater extent than the more employable subgroups. 

Among applicants (see Table 3.^), there was a clear tendency for wbmeh 
with characteristics that appeared to be barriers to employment to have 
participated in job search at least as much as, if not more than, those who 
appeared more employable; There were no differences among applicant 
subgroups in measures of participation in work experience, but the picture 
was different for training and educational programs. For these activities, 
subgroups with the greater barriers to employment had similar or lower 
participation levels thaa the more employebie ones, except for the 
applicant group who had been dependent on welfare for two years or more in 
the past. These women were significantly more likely to participate in 
training than women with less prior welfare history. 

Partlclpatlbh among recipient subgroups (as seen in Table 3.5) 
followed the general applicant pattern: the less employable recipients 
were more likely than, or as likely as, the more employable groups to be 
active m job search, but as was the case for applicants, this pattern was 
less evident for work experience and education and training. in 
particular, recipiients without a high school or GED diploma and those who 
were 30 years or older were significantly less likely to be enrolled in 
education or training programs than those who had completed high school. 

While these overall patterns suggest that staff tried to encourage the 
subgroups that potentially facied greater barriers to participate in program 
activities as much as the more employable groups. Tables 3.i| and 3.5 show 
that they were most successful in job search. This is hot surprising 
because ESP staff as a whole were convinced that job search was an 
appropriate mandatory activity for almost all people on welfare. 



staff had mixed views, however, about encouraging the most hard-to- 
emplby groups to take part in education and training, believing that iany 
had hot yet built up the confidence to handle these activities. in 
addition, some enrollees lacked the educational or skills backgrounds to 
succeed in these courses or vocational programs, and the staff found that 
some of these enrollees simply could not meet the requirement of the 
services to which they were referred. The more disadvantaged groups also 
were more likely to fail to follow through on the referrals by ESP staff. 

V. Partlf^ipat.inn 



Figure 3.2 shows the cumulative participation rate for all ESP 

enrollees, as well as the rates for enrollees randomly assigned in the 

_ _ _ _ _ . • 

early and later months of the year-long sample enrollment period. 

(Information on the participation patterns of early and later applicant and 
recipient subgroups is presented in Table 3.6.) As seen in the figure, 
women randomly assigned through March 1984 were tracked for a 15-mbnth 
follow-up period, while the group entering the sample from April through 
September 1984 had at least nine months of follow-up. 

Two conclusions emerge from Figure 3-2 and Table 3.6. First, for the 
total sample and both subgroups, the participation rates continued to climb 
through the ninth month after random assignment. This is because staff 
initially deferred some enrollees from job search but reassessed them 
periodically; as their situations changed, the enrollees were often 
assigned to the components to begin their search. 

The other notable finding is the somewhat lower participation level of 
the later group of experimentais, perhaps caused by the different 
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FIGURE 3.2 
VIRGINIA 



ALL AFDGi CUMOLATIVE PiTICIPATION RATES FOR 
EXPERIHENTALS, iT PERIOD OF RANDOM ASSIGiENT 
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SOURCE! ^alcnlations from the Virginia Autbrated Client Information. System and from program activity data 
collected from case file records of a random stlbsample of experimentals in Fairfax County. 
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composition of the two samples. As notod In Chapter 2, the agencies had 
randomly assigned their existing recipient caseloads by the end of March 
198i*. Recipients randomly assigned after that time were those who had just 
been newly determined WIN-mandatory ^ usually because their youngest child 
had become school-age. Thus, the earlier group of women — a group 
containing primarily the existing WIN-mandatory caseload but also a mix of 
other eligibles — may be in fact more typical of the enrollee pool in ah 
ongoing program than the later sample dominated by applicants and the newly 
determined mandatory recipients. the two groups had many different 
background and demographic characteristics — differences that could have 
Influenced their participation levels. 

To determine if the behavior of applicants or recipients unduly 
Influenced these overall rates, the data In Table 3.6 were compiled to 
analyze their separate performance Indicators In the early and later 
periods during a uniform nine-month follow-up period. As seen in this 
table, the applicants randomly assigned after March 198i{ were as likely as 
the earlier group to participate in job search and work experience, 
although less likely to take part in education or training programs. In 
contrast, the later recipients were less likely than the early recipient 
group to participate in job search, as well as education and training. 
Thus, the later recipients* perforaance largely accounts for the overall 
lower activity levels of the later groups 
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CHAPTER ^ 
PROGRAM IMPACTS 

I. Introduetion 

Two questions are basic to answer in determining the effects of any 
program. First, what happened to those enrolled In the program? In other 
words, given the outcomes of interest — In this case, employment and 
welfare what were the average outcomes among enroliees? Second, what 
would have happened to the enroliees if the program had not existed? The 
first question Is answered by examining the behavior of all program 
enroliees, or the experimental group. The second is answered by following 
the behavior of a control group, which is similar in background and demogra- 
phic characteristics to the experimental group but not eligible for program 
services. The effects, or Impacts, are the differences between the average 
outcomes for the two groups. 

This chapter presents the results of the experimental design impie- 
mehted to evaluate the Virginia Employment Services Program. Chapter 2 
explained how sample members from the target population were assigned at 
random to either the experimental or control group. Data were collected 
that mearured the employment rates and earnings, as well as welfare receipt 
and payments, of the two groups to detennlne If ESP improved the empibjrment 
prospects of members of the experimental group and reduced their welfare 
receipt, compared to those outcomes among controls. 

Chapter 2 verified that measured differences between the two groups in 
background and demographic characteristics were slight, and that the 
average outcomes for controls should reveal what would have happened to 
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experlmehtals had program assistance not been available, this is extremely 
Important slnce^ even without program services, many individuals find 
employment on their own or leave the welfare rolls for othpr reasons in a 
relatively short period of time. Use of a control group allows hon- 
program-induced placements or normal welfare departures to be subtracted 
from the levels of prograHH- produced achievements: i.e., higher levels of 
employment or lower rates of welfare receipt. 

Overall, this chapter will show that, for the full impact sample of 
3,182 applicants and recipients,*^ there were modest gains in employment for 
the experimental group in the short run. Cumulatively, over nine months, 
or three quarters, experimentals were 3.3 percentage points more likely to 
have been employed at some time than controls. While this increase was 
statistically significant, a small earnings increase was not. Welfare 
payments to experimentals also went down by a statistically significant 
amount during a one-year period, with the largest reduction occurring in 
the third quartern However, the proportion receiving welfare remained 
unchanged by the program in this first year. 

As noted in other chapters, the 1 ,285 sample m'^Tibers who were appli- 
cants at random assignment and the 1,897 who were already recipients were 
analyzed separately, as were other distinct subgroups. These subgroup 
analyses show the importanoe of disaggregating the results for the full 
sample. in the short term, employment impacts for the full sample came 

almost entirely from the program's stronger gains for applicants, not 

^ _ _ _ 

recipients. (Earnings impacts were not statistically significant for 
either applicants or recipients.) And, while the program produced short- 
term welfare savings for the recipient group, the savings were not 
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sustained In the longer run. 

Particularly among applicants, longer-term program effects for ah 
early group, followed for six quarters, continued the story of the full 
sample's short-term gains. The two additional quarters of follow-up showed 
continued employment impacts that were statistically significant in the 
fbiarth, fifth and sixth quarters, as well as modest but significant welfare 
savings over the 18 months. Welfare savings for recipients, on the other 
hand, declined markedly in the fourth and fifth quarters, and employmant 
gains remained minimal. 

t 

!!• ^i^alvsis Issues^ and Data Sources ; 
A. ^ecei^ of Program Services > 
Chapter 3 deacribcd the sorvices available to the 2,138 experlmentals 
In the research sample. it also made clear that not everyone assigned to 
the experimental group actually participated in ESP, although everyone 
assigned to experimental status was included in the calculation of average 
outcomes. Table 3.2 shows that 58.3 percent of all experlmentals — 
including 41.7 percent of applicant experimentais and 69.4 percent of 
recipient experlmentals — were counted as having participated in a program 
component. To the extent that nbnparticipants were not affected by the 
program^ the inclusion of these experimentais may have diluted the Impact^s 
attributable to participating experimentais.^ 

Very few of the 1,044 controls were served by the ^ployment Servic&s 
Program; by design, none were supposed to be. However, on their own, sub- 
stantial numbers of controls did enter and receive education and training 
services, many of the same types of services to which experimentais were 
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referred by ESP staff; Although It was recognized (Chapter 3) that these 
empioyment and training services were available to all residents of the 
community meeting the various eligibility requirements and that anyone 
could enroll — the number of controls doing so was not expected to be 
large. However, according to a special study involving a small subsample 
of controls J 15 percent of control recipients and 9 percent of control 
applicants did take part in training and education activities without 
referral assistance from ESP staff. 3 (See Table 4.1.) These levels were 
only 3.3 and 2.6 percentage points below the participation rates of 
experlmentals. Thus^ the expected experimental-control differential in 
education and training did hot materialize, partly because of the lack of 
additional resources for agencies to provide such services, as well as 
their inability to forge difficult linkages quickly with outside education 
and training providers. 

An Important finding is that ESP was thus not able to substantially 
Increase the receipt of education and training resources for the targeted 
population beyond that which they would have found on their own, and that 
the program effects discussed in this impact analysis could not have 
resulted from education and training services alone. While these services 
may well be effective for welfare groups and lead to promising outcomes, 
this study cannot address this issue. 

B. Caseload Turnover Versus Impa cts 

A look at the average characterlsclcs of the full sample reveals iti 
potential employment barriers. (See Table 2.3-) Approx-lmately 60 percent 
had spent more than two yeara on welfare in the past. Some 63 Percent had 
not been employed during the previous year, and more than half had not 
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Any education or training is a combined, unduplicated count of the above ftfee MtBgoriaB, 
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received high school dipibmas or GEE) certificates. On the other hahd^ 
about ^Q percent of the fuii sample consisted of new applicants or re- 
applicants for welfare. More than half of this group had held a job at 
some time in the year prior to this current welfare application. (See 
Table 2.i|.5 

It Is thus Important to realize that dependency is not a permanent or 
even a long-term situation for many families. Within a short period of 
time, substantial numbers of welfare recipients find employment or leave 
the welfare rolls for other reasons. This pattern is exemplified by the 
typical behavior of controls: the applicants and recipients who were not 
eligible for the ESP array of services. 

Three points stand out in an examination of control behavior. First, 
almost one- third of the control applicants never received welfare during 
• the year after enrollment, many of them because their welfare applications 
were denied. Second, over this one-year period^ the prop>ortlbn of appli- 
cants who did receive welfare declined from 61,3 percent to 1|0,1 percent, 
and that of reoipients from 97.6 percent to 73-3 percent,^ Thus, even in 
the absence of the program^ a substantial number of both applicants and 
recipients were never oh or had left the rolls, third, while some propor- 
tion of any welfare population will find jobs without the assistance of 
program services, not all people leave welfare because they have found 
jobs. During the fourth quarter^ 29.5 percent of control applicants were 
employed and not on welfare. However, a similar proportion (28,6 percent) 
were not receiving welfare but did not have jobs. The rest were still 
receiving welfare, and a small proportion (about 10 percent) were holding a 
job from which the income was so low that they still qualified for 
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benefits. (See Appendix Table F.I.) 

The control recipient story Is quite different. For example, recipi- 
ents were much less likely than applicants to be off welfare and/or 
employed. Only 12.3 percent were both off welfare and employed; I3,d 
percent were off welfare but riot holding a job. Almost three-quarters were 
still on welfare one year after random assignment. (See Table F.I.) 

e. Combined Ebcperlmental Groups 

It should be restated here that, as discussed Iri Chapter 2, the 
research design was developed to evaluate not only the full array of ESP 
services but also a specific sequence of job search followed by work 
experience. The Intended difference between the sequences was to be the 
availability of training and education for the All ESP Services group. As 
noted in Chapter 3, however, the difference between the two experimental 
groups' participation in education or training programs was small; there 
was, consequently, no reason to expect that the Impacts for the two program 
sequences would differ substantially. The conclusion reached In Chapter 3 
was that, except in a few local agencies, both experimental groups received 
fairly similar program services, and any differences were not large enough 
to justify separate evaluation of each sequence. (In fact. Impacts were 
estimated according to the original design and, with one exception, there 
were no statistically significant differences between the two experimental 
groups. )5 

D. ^Organization of Data Sources 

Chapter 2 pointed but that the basic data sources for the Impact ana- 
lysis are records data: state and county AFDC welfare records of benefits 
paid to recipients, and earnings data that employers report to the state 
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Unemployment Insurance system • While AFDC records are kept on a monthly 
basis, tJi data are recorded on a calendar quarter basis: that Is, In 
three-month periods beginning in January, April, July and October, 

I^ius, calendar quarters are the point of reference In the discussion 
of employment and earnings, but for AFDC receipt and payment levels, the 
first quarter of foilow-up begins in the month of random assignment. Thus, 
except where indicated, the quarters covered by the two sets of data do not 
exactly correspibnd.^ 

III. ^brt - T e m -I tDpact ^-^^^^^T^^ -Sample 

Table 2J,2 presents the short-term Impacts'^ of ESP on the fuli sample 
of applicants and recipients. Major outcomes examined are empibyment 
rates, average earnings^ the proportion receiving welfare, and the level of 
welfare payments. One point Is important to note before considering the 
following sections. Follow-up Is nine months for employment and earnings; 
one year for welfare outcomes, in accordance with the data organization 
mentioned above. In addition, because the first quarter of empioyraent can 
Include job activity and earnings changes prior to a person's entry into 
the sample, first-quarter employment and earnings impacts are not consider- 
ed true fbllow-up Impacts, Impact quarters for these two measures are 
quarters two, three arid fbur^ and cumu^Ltive impacts are based only on 
quarters two thrbugh fbur. 

Overall, the ESP prbgrain produced a modest 3,3 percentage point 
increase in the prbpbrtibn of experimentals employed at any time duririg the 
second, third or fourth quarters. This impact was statistically slgnifl- 
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Att AFOC: IMPACTS OF THE EHPLDYMENT SERVICES PROGRAM 
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the Bonth of randOB aaslgnBsnti 

^Quartar 1p the querter of random asslgnmenti may contain some eernlngs 
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cant; When the data are examined quarter by quarter^ the Impacts can be 
seen to Increase slightly, reaching statistical significance In the third 
and fourth quarters. In quarter three, the experimental employment rate 
was 3.3 percentage points higher than the rate for controls, 27.9 percent, 
in quarter four, the experimental rate had risen to 3^1.^1 percent; the 
impact of the program had risen to 3.9 percentage points, which represents 
a 12.8 percent gain over the control mean of 30.5 percent. There were no 
statistically significant earnings differences between the two groups, 
although average earnings for experimentals rose above the control mean of 
$1,038 by $81 during the follbw-up period. 

There were also no significant changes In the proportion of experl- 
mentais receiving AFDC income, either cumulatively or by quarter. Over the 
four quarters, 86.0 percent of experimentals versus 86.1 percent of 
controls ever received welfare payments. this sxaggests that the program 
did not deter individuals from continuing with their welfare applications, 
or from receiving benefits. 

In contrast, cumulative welfare payments to experimentals ($1,923.28) 
averaged about $84 less than cumulative payments to controls ($2,006.87). 
Reductions were statistically significant in quarters two and three, with 
welfare savings of $2^1 and $30, respectiveiyi This trend toward Increased 
savings was reversed In quarter four, where the difference of only $20 vras 
not statistically significant. 

Generally, however. It can be concluded that the small gains in 
employment for experimentals were accompanied by modest reductions in 
welfare grants, although, in the short term, the program did hot succeed in 
reducing the caseload. 
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IV. Impacts Among Appilcants 

As noted previously, impacts were analyzed separately for thie two 
important subgroups of the full sample — people who were either new 
applicants for welfare (or re-appiyirg for aid), and those who were 
recipients, whether already mandatory or newly determined mandatory 
(usually because their youngest child had become school-age) when they were 
randomly assigned. (See Appendix Tables F.7 and F.8.) Differences in 
demographic and background characteristics of the two groups and their 
somewhat different participation rates in the various ESP components had 
suggested that separate analysis could reveal much about the effectiveness 
of the ESP strategy for different groups of the welfare population, 
particularly for the existing longer-term recipient caseload. This group 
has not been studied in other programs as yet evaluated in the MBRG 
demonstration. 

This analysis of 1,285 applicants and 1,897 recipients focuses first 
on thie impacts of each group during the short-term and then follows the 
trends through an additional two quarters using the early portion of either 
the full applicant or recipient sample: those persons who were randomly 
assigned between August I983 and the end of March 198^1. The early sample 
is described in more detail belbwi in Section B. 

Table I1.3 shows that^ among applicant experimeritals, employment had 
increased to a greater degree than for the full sample of experimentals: 
Cumulatively, 59.5 percent of the experimentals were employed over the 
nine-month follow-up period compared to 5^.1 percent of controls, for a 
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AFDC APPLICANTS; IMPACTS OF THE EMPLOYMENT SERVICES PROGRAM 
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Ever Received Any AFDC PeymentB (X] 
Quarter of Renddm AeBlgnment 
Querter 2 

Quarter 3 ' 

f\ » m tm ^ ^ Jfc ^ A 

wuBrbBr H 


61 i8 
58.6 
44.7 
37.8 


61 .3 
56 .2 
45 .7 
40 .2 


+ 0.4 
+ 2.4 
"1 .0 
"2.4 


Average Totet AFDC PeymehtB 
Racalvadi QuartorB 1-4 [$] 


12D2.46 


1277.68 


-75.22 


Average AFDC Peytnente Received (1] 
Quarter of Rendolh AeBlgnment 
Quarter 2 
Quarter 3 
Quarter 4 


315.78 
368.74 
274.81 
242.02 


314.51 
384.60 
307 .77 
270.70 


+ 1.28 
"14.95 
-32.66 
-28.68 


SampL e Size 


857 


428 





SOURCE AND NOTES: See Tebte 4,2, 
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statistically significant 5.^ percentage point Impact. Thie average number 
of quarters experimentals were employed Increased by aimost 13 percent, 
from 1.17 to 1.32 months. Quarter by quarter, experimental employment rose 
from a level close to that of controls In the first two quarters so that, 
by quarter four, percent of experimentals were employed versus 39.6 

percent of controls, for a statistically significant impact of 7o2 
percentage points. (See Figure i|.1.) 

Earnings reflected the almost equal experimentai-cohtrbi employment 
rates in the first two quarters, with experimentals initially earning 
somewhat less than controls. impacts went from negative to positive, 
turning upward and increasing: the cumulative nine-month impact, of $127.30 
was an 8.5 percent increase relative to the control mean of $1,50i*.57. The 
highest gain was registered in quarter four^ but none of the earnings 
differences were statistically significant, perhaps because this measure 
varies to a greater degree person by person than other outcome measures, 
thus making it difficult to detect differences that are meaningful — that 
is, statistically significant ~ and not due to chance. 

ESP did not result in statistically significant reductions in the 
proportions of applicants receiving welfare during the one-year follow-up 
nor in significantly lower levels of welfare benefits. More than two- 
thirds of both experimentals and controls received some AFDC payments 
during this time, with both groups spending on average about two- fifths of 
the year on welfare. Controls received about $1 ,280 in welfare payments 
over the 12 months, and welfare savings for experimentals averaged only 
$75. Nevertheless, while none of the differences was statistically 
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FIGURE 4.1 



VIRGINIA 



AF0E APPLICANTS: TREN0S IN QUARTERLY EMPLOYMENT 
RATES AND AVERAGE AFDG PAYMENTS 
(AUGUST 1983 - SEPTEMBER 1984 IMPAGT SAMPLD 
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SOURCE: See Table 4.3 
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significant, welfare reductions were isomewhat larger during the last 
quarters of follow-up. (See Figure 

Bi Composition of the Earlv Sample 

Of the 3,182 members in the full sample, 2,301 were randomly assigned 
before April 198^1 and thus made up an early sample for whom six quarters of 
folidw-up data were collected. Within this sample, 744 were applicants; the 
rest (1,557) recipients. Composition of this early sample differed from 
the later one because, as discussed in Chapter 3, the local agencies had 
enrolled their existing recipient caseloads into the sample first, along 
with any current applicants and newly determined mandatory recipients. 
Thus, while the applicant portion of the early sample is representative of 
the later applicant sample, the characteristics of the early recipient 
sample should be quite different from those of the later recipient sample. 
Later recipients were more frequently those newly determined mandatory, not 
ones drawn from the existing caseload. 

The imporcance of the longer foiiow-up should be noted again. Because 
the Employment Services Program was to be a multi-component sequence of 
services, it could take some time for sample members who participated in 
more than one component to do so. Reassessment after each component could 
further deliiy the process. In addition, educational prograias and training 
courses are not short-term activities, and, even though this evaluation 
cannot address their relative effectiveness, the extended fbllbw-up allows 
the maximum amount of study. 

C. l^ngep-^T ^rffi^bE^icaiiit la^^ 

As shown in Table 4.4 and Figure 4.2, employment increased among early 
applicant experimentals over the longer-term (i.e., 18 months or six 
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TABLE 4.4 
VIRGINIA 

AFDC APPLICANTS; LiDMER-TiRM IMPACTS OF THE EHPLOVMENT SERVICES PROGRAM 
(AUGUST 1983 - MARCH 1984 IMPACT SAMPLE) 



Dutcbme and FoLLbw-Up Period 


Experimentale 


Cdntr oj^s 


Dif_f e_cince 


Ever inptoyedy Quartere 2 6 [%)° 


66.7 


62.8 


+ 3.9 


*veraflB Number, of Quart eriaW 1th 
ERpLoynenty Quarters 2 - 6 


2. SI 


1 .93 


+0.28* 


Ever EHpLdyed [%] 

PM^r^f^r pf RandOBi Asalonment 
Quarter 2 
Quarter 3 
Qoarter 4 

Q ue r te r 6 


43.5 
37.3 
44.5 
47.7 
46 .1 
44.9 


44.6 
38.5 
36.7 
40.8 
37.6 
37.4 


-1 .1 
-1 .3 
+5.8 
+ 6.9* 

+e.8** 

__ +7i&*'^ 


Average Total Eerhfhgsv Querters 2"-6 (Sj^ 


2884. 96 


2463 .49 


+401 .49 


Average Total Earnf ngs ($) 

Quarter of Random Aselgnment 

Quarter 2 

Quarter 3 

Quarter 4 

Quarter 5 

Quarter 6 


358.69 
366.06 
535.22 
630.38 
640.20 
691 .10 


429.20 
400.05 
482.31 
540 .65 
535.36 
525.10 


- 70.52 

- 11 .97 
+ 52.91 
+89.73 
+104.81 
+166.01** 


EyerRocelved Any AFOC Paymente, 
uuarbere i — d i%] 


72 .2 


75.1 


--2.9 


Average Number of Months Receiving 
AFDC Peymenter Querters 1-6 


6.96 


7.74 


-D.77 


Ever Received Any AFOC Paymente [%] 
Quarter of Random Assignment 
Quarter 2 
Quarter 3 
Quarter 4 
Qiierter 5 
Quarter 6 


63.1 
60.1 
47.7 
39.4 

*9 7 - O 
i3 / . d 

32.4 


64.7 
61.5 
50. b 
43.2 
42 .0 

40.8 


-1 .6 
-1 ^3 
-2.3 
-3.8 
-4.6 

_-i-4** 


Average TdteL AFDC Paymente 
Receivedy Quarters 1 6 [1] 


1677.03 


1 908.59 


-231 .56* 


Averege AFDC Peymente Received [S] 
Qoartar of Rendom Assignment 
Quartar 2 
Quarter 3 
Qoerter 4 
Qoarter 5 
Quarter 6 


316.12 
376.25 
263 .01 
250 .44 
241 .51 
209.69 


324.00 
406.88 
327.32 
297 .07 
228*26 
275.06 


- 7.88 

"30.63 

"44.31 

-46.64* 

"36.75 

"65.36** 


Sen p t a 6 i z a 


492 


^51 





SDURCE AND NOTES: See Table 4.S. 
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FIGURE 4.2 
\7IRGINIA 



AFDC APPLICANTS: TRENDS IN QUARTERLY EMPLOYMENT 

RATES AND AVERAGE AFDE PAYMENTS 

UUGUST 1983 - MARCH 1984 IMPACT SAMPLE) 
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quarters of follow-up), basically following the four-quarter trend (Table 
4 •3). In the last three quarters of the six-quarter follow-up^ employment 
Impacts were 6.9, 8.6 and 7-5 percentage points, respectively, with each of 
these Impacts statistically significant. Essentially, employment rates for 
both applicant experimental and control groups had taken similar paths ov^r 
the l8-mbnth period. From a low In quarter two, the rates of both groups, 
especially the experimehtals, increased in quarters three and four^ and 
then dropped lower in the last two quarters. As Table shows, the 

experimentals stayed ahead of controls after quarter two. At a not stat- 
istically significant 3,9 percentage points^ the cumulative impact was 
similar to that for the full sample, but lower than the significant 5.4 
percentage point gain of the full applicant sampile. 

Quarter-by-quarter earnings impacts closely paralleled employment 
impacts, with the difference negative in the second CiUarter but then 
turning positive and continuing to increase slightly. The earnings gain in 
the sixth quarter was $166.01, a statistically significant amount that was 
32 percent higher than the control group* mean of $525. 

Also, in the iS^month period, the program reduced the cumulative 
amount of welfare bene^^its paid to applicants by a statistically signi- 
ficant $2?1.56, a 12 percent difference from the control mean of $1^908.59. 
The impacts were fairly similar quarter by quarter (except for quarter one) 
until the sixth quarter, when the statistically significant reduction of 
$65.36 represented a change of 24 percent from the control group mean of 
$275 •06. The difference in the proportion receiving welfare both cumula- 
tively and by quarter remained relatively unchanged by the program until 
the sixth quarter. At that point, the 8,4 percentage point difference in 
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the percentage of people receiving any payments was statistically 
significant and showed that, for the first time, ESP had had an effect on 
the size of the caseload. 

V. lapacts Among ReGlpients 

This section presents Impacts for the 1,897 sample members who were 
recipients when they were randomly assigned — both those who were already 
mandatory as well as those newly determined mandatory during the period of 
random assignment. Short-term Impacts over four quarters are presented 
first In Table 4.5^ while Table U.S and a discussion of longer-term impacts 
for the early sample of recipients follows. 

A. Short-T^sria 4tectfil^nt^I#t>a^^ 

As seen In Table 4.5^ the program produced a small employment gain for 
recipient experimeritals, amounting to 2.2 percentage points. This gain, 
based on a control group mean of 31.1 percent, was not statistically 
significant. Quarter by quarter, the size of the Impact declined from a 
statistically significant 3.4 percentage points in the quarter of random 
assignment to a nonsignificant 1.8 point difference at the end of the 
follow-up. (See Figure 4.3.) The first-quarter gain in this case is worth 
noting. While quarter one is not usually considered a true Impact quarter 
for employment — and should not be so construed for the applicant sample 
— there was so little prior employment among recipients that the first 
quarter may in fact reflect an immediate program Impact — one that 
deteriorated rapidly. 

While the absolute average earnings of both experlmentals and controls 
improved over the four-quarter period, there were no statistically slgnlfl- 
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TABLE 4.5 



VIRGINIA 

AFOC RECIPIENTS: IHPACTS OF THE IMPLOYMENT SERVICES PROGRAM 
(AUGaST 1983 - SEPTEMBER 19B4 IMPACT SAMPLE) 



Outcove arsd Fottdv-Up Period 


Fstmip^Hiinfiif c 


uonbTOi 6 


ui ft erence 


Ever Enptoyedv Quartere 2-4 


33.4 


31.1 


+2.2 


Average Nuaber of Qsercere With 
Eiiploy«ent» Qaarters 2-4^ 


0.68 


0.62 


+0.06 


Ever Enptoyed l%] 

Uuarter of RandoB Asslgnnent 
Quarter 2 
Que r te r 3 
Quarter 4 


14.7 
19.7 
22.8 
26.0 


i 1 .3 
17 .4 
20 .9 
24.2 


+3.4** 

+2 • 3 
+ 1 .9 
+ 1 .8 


Average Total EarnlngSv Quertere 2*4 ($]^ 


776.12 


713.82 


+62.30 


Average totat Eernlnge ($j 

Quarter of Rendom Asslgninent 
Quarter 2 
Quarter 3 
Querter 4 


103.66 
181 .62 

CD U • 0 O 

323.72 


77.51 
166.41 
241 .86 
303 .55 


+26.15 
+23 .41 
+18.72 
+20.18 


Ever Received Any AFDC PayoientSv 
Querters 1 - 4 t%j 


97.2 


97 .8 


-O.G 


Average Nuaber of Months Receiving 
AFOC PeynentBi Ctoarters 1-4 


9.64 


9 .81 


-0.18 


Ever Received Any AFDC Paymehte (%j 
Quarter of Rendom AsslgrtmenC 
QoB r te r 2 
Qoerter 3 
Quart6r4 


96 .9 
88.2 
80 .1 
74.4 


97.6 
90 .3 
82 .4 
73.2 


-0.7 
-2.1 
-2.3 
+ 1 .2 


Averege Totat AFDC PaymencB 
Hecelvedi Quertere 1-4 ($] 


2407.80 


2508.79 


-100 .99** 


Average AFOC PeymentG Received ($j 
Quarter of Random Aeelgnment 
Quarter 2 
Quarter 3 
Quarter 4 


695 .30 
625.1 8 
563.95 
523.36 


712.83 
658.36 
586 .32 
541 .28 


17.53 
-33 .1 8** 
-32. 37** 
-17.92 


Sanpte Size 


1281 


616 
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TABLE 4.5 (conti nued j 



-rnfnoS''rL " [ ? " ^ » ' < ° °» » " t * h of Virginie Unemployment Insurance 

•""^•'^^""••as, welfare records fron, the Vlrglnle Auto«eted CUent Information 
Syatem and Fairfax County AFDC case fltes; 

NOTESs These data Include zero walBes for aanpLe aSnbers not employed end for 
ea«pt. .a«bar. not reca < v < n g « el f a r e. These date ara r a g r e s si o n-a dd u ste d u si n g 
ordinary teast squares, controlling for pra-renddm assignment characteristics of 
.nrdrffe'L^c:;. '° ^"^-^ discrepancies In calculating su«e 

-hi K . I^^ quarter of random aeeignmant refers to the calendar quarter durinq 

•hlch an individual entered the aample for employment and earnings data. For AFDC 
pey.anta, the quarter of random assignment refers to the three months beginning „ith 
the Month of random assignment. 

--- Beceuso of underreporting or imnedlete termination of welfare benefits, 

the percantaga of racipients racSiving AFDC payments in the querter of rendon: 
eeelgnnent cfbeis hot equeL 1D0« 

frn„ .h. ^' the quarter of random aBBlgnment, mey contain some earnings 

from tha period prior to rendom ascf^.K^ent and is therefore excluded from meesures of 
total fotto«-ap for emplbyment and e'.'r.ilngs. 



A 



, , tse-teiled t-test .es appilad to differences between experimental end 

control B.^oaps. StetisCical significance levels are indicated es: • = 10 percent- 
= S parcant; = 1 percent. ' 
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TABLE 4.6 



VIRGINIA 



AFOC RECIPIENTS: LONGER-TERM IMPACTS OF THE EMPLOYMENT SEHVlCiS PROGRAM 
(AUGUST 1983 - MARCH 1984 IMPACT SAMPLE) 



Outcdihe end FblLbw-Up Period 


ExperlmentetB 


£oiLt_ro t s 


Oifference 




40 • 9 


37 . 2 


i^3 .6 


Average Nuaber of Que rtere^WI th 
Enpldyaentf Quertera 2-6 


1 .20 


1 .08 


+ 0.12 


Ever Employed. (Xj 

Qoerter df Rendon Aaalgnaent 

Quarter 2 

Qua r te r 3 

Quarter 4 

Querter S 

Qoerter 6 


14^3 
IB. 9 
22.5 
25.8 
26 .6 


10.7 
15.8 
19.9 
23 .4 

26i2 
22.3 


+3.6** 
+ 3.1 
+ 2.5 
+ 2.5 
+ 0.5 
+ 3 .4 


Averege Total iernlngBf Qaartere 2-6 


1486 .79 


1310.22 


+176.56 


Average ToteL Eerhlngs (1] 

f^Marter of Random Aaalgnment 

Quarter 2 

Qoarter 3 

Quarter 4 

Quarter 5 

Quarter 6 


99.55 
137.40 
262.84 
320.09 
361 .33 
355.14 


70.13 
152.51 
225.07 
2 84.3 3 
341 .89 
306 .43 


+29.42 
+34.89 
+37.77 
i-35.76 
+1 9.44 

+48.70 


Ever Received Any AFDC Paymentif 
Quarters 1-6 [X] 


97.5 


97.8 


-0.3 


Average Number of Month 6 Recisi v 1 ng 
AFOC PayaentBf Quarters 1-6 


13 .49 


13.56 


-0.07 


Evcir Received An* AFOC Peymente l%] 
Quarter of Random Aeelgnment 
Querter 2 
Querter 3 
Que r t e r 4 
Qoerter 5 
Qua r t e r 6 


97 .1 
88^7 
eo .4 
74.4 
69.5 
66.0 


97. 
91 .0 
82 .8 
72.8 
66 .3 


-Di5 
-2.3 
-2.3 
+ 1 .5 
+ 3.2 

T c . u 


Average Totet AFPr Peymsnts 
Received* Quarters 1-6 ((] 


3319.45 


3401 .80 


-82.35 


Average AFDC Peymente Received ($] 
Quarter of Rendom Aeslgnment 
Quarter 2 
Querter 3 
Querter 4 
Querter 5 


691 .16 
621 .67 
560.18 
519.61 
476.81 
45D.D3 


705.68 
650 .23 
589 i3S 
530.06 
470.58 
455.88 


-14.52 
-28.56* 
-29.20* 
-10.45 
+ 6.23 
" 5.85 


SanpLe Size 


1D5Q 


507 





SOURCE AND NOTES: See Table 4.5. 
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FIGURE 4.3 
VIRGINIA 



AFDC RECIPIENTS! TRENDS IN QUARTERtY EMPLOYMENT 
RATES AND AVERAGE AFDC PAYMENTS 
(AUGUST 1983 - SEPTEMBER 1984 IMPACT SAMPLE) 
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cant Impacts for experlmentals in any quarter. The cumulative earnings 
impact over quarters two through four was $62^30, a difference that was 
also hot statistically significant. 

In contrast, recipients were paid lower benefits over the short term, 
With significant welfare savings over one year of $100 .99* Quairter by 
quarter, however^ the data show that most of the savings came in the second 
and third quarters. By the fourth quarter, the experimental-control 
difference had dropped to the not significant level of quarter one. Thus, 
welfare savings that recipients realized In the short run did not seem to 
endure. The proportion receiving welfare remained essentially the same for 
both groups. 

B. Longer-TeriD Reclt>l^nt^lBmae%a 

Table ft. 6 shows the Impacts for the 1,557 recipients who yere randomly 
assigned before April 198ft, a group drawn primarily from the existing 
recipient caseloads of the agencies. Like the full sample^ the early 
recipient sample experienced Immediate emplbym^^ht gains In quarters o-j-j and 
two, although these declined until quarter slx^ when the Impact again 
reached the over 3 percentage point level of the first two quarters. (See 
Figure ft. ft.) Only the first quarter Impact was statistically significant^ 
however. 

The cumulative empioyment Impact of 3.6 percentage points was not 
significa/it for recipients, nor was the cumulative earnings Impact. The 
cumulative earnings advantage for experlmentals was In excess of $175 
more than 13 percent of the average for controls (at about $1^300). 
However, none of the earnings Impacts ~ either quarterly or cumulative — 
was statistically significant. 
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FIGUPE 4.4 
VIRGINIS 



AFDC RECIPIENTS; TRENDS IN QUARTERbY EHPhGYHENT 
RATESANO AVERAGE AFDC PAYMENTS 
(AUGUST 1983 - MARCH 1984 IMPACT SAMPLE) 
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In line with these early small employment and earnings gains, welfare 
payments for experimentals declined In the second and thlrcJ quarters, with 
the ImiDacts in those quarters statistically significant. By quarters five 
and six, however, there were no longer any impacts at all; this pattern 
mirrors that of the full recipient sample, except that the early recipi- 
ents* quarterly welfare savings were somewhat Jailer, The proportion 
receiving welfare in this early sample did not change significantly. 

In light of these findings, it seems reasonable to conclude that ESP 
exerted little enduring influence on the employment and welfare behavior of 
recipients, particularly those who had been oh the caseload for some time. 
Early favorable signs of reduced welfare dependency did not hold up over 
the longer term. Given the stronger findings for the applicant sample, 
both in the short- and longer-term, the program's significant impacts on 
the full sample apF>ear to stem almost exclusively from the larger impacts 
on applicants. However, It will be Important to examine other subgroups of 
the full sample, as well as the benefit-cost analysis in Chapter 5, which 
takes a broader view of the subgroup findings i 

VI. 4?rban ahd-Ruf*al Imbac trg 

Some 2,507 sample members^ or 78.8 percent of the total sample, came 
from one of the four agencies In the urban areas of northern Virginia. In 
contrast, 675» or 21.2 percent of the full sample, were randomly assigned 
from one of the seven agencies located in the predominantly rural counties 
of southwestern and south central Virginia. Because local labor market 
conditions varied, as did state payment standards and sample members* 
demographic and background characteristics, the urban and rural sanp>les 
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were examined separately over the short term to see if ESP worked 
differently In different environments. 

Tables 4.7 and M.B show the resultis of separate impact estimates for 
the two samples. Table 4.7 shows that employment impacts for the urban 
sample were somewhat larger than those for the total sample. The cumu- 
lative employment Impact for urban experlmentals was a statistically 
significant 3.8 percentage points^ with quarterly employment gains 
increasing from 3.3 percentage points In the second quarter to a signi- 
ficant 4.5 percentage point gain in the last quarter ^ on a base employment 
rate of 31.9 percent for controls. 

Welfare savings were also statistically significant In quarters two 
through four and for the four quarters cumulatively. Over the one-year 
period, urban experlmentals, compared to urban controls^ received $111 less 
in welfare payments, going down from a control average of $2^085.71. This 
may reflect the earnings Impacts In the last two quarters, which were^ 
however, not statistically slgnlflciant. 

It appears that ESP was not as effective In the rural agencies, 
although, because of the small sample sizes, It was difficult to assess 
results. Table 4.8 reveals no statistically significant Impacts on any 
outcome measures and no clear trends. When the sample was further sub- 
divided Into rural applicants and rural recipients (as seen In Table 4.9 
and 4.10), not surprisingly, only one of the impacts was significantly 
different from zero. 

In contrast, there were notable Impacts for the urban applicants in 
employment (Table 4.9). While there was no significant cumulative impact, 
the employment effects rose significantly in quarters three and four to 
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TABLE 4.7 



VIRGINIA 



URBAN APPLICANTS AND RECIPIENTS! IHPACTS OF THE 
EMPLOYMENT SERVICES PROGRAM 
(AUGUST 18B3 - SEPTEMBER 1984 IMPACT SAMPLE} 



Outcone and Fol to9-Up Perl od 


Expe r 1 ne n ta L e 


Control 8 


01 ff e rence 


Ever Enptoyedt Qaarters 2-4 1%]^ 


46 .8 


43 .0 


+ 3.8* 


Avarago Nanbar of Qua rte rs W 1 th 


1 . 0 U 


0.9 6 


+0 .1 2** 


Evar Eaptoyad (X] 

Quartar of Random Aeelsnftient 
Quartar 2 
Quartar 3 
Quarter 4 


2 8.3 

30.5 
33.3 
36.4 


27 .5 
27 .2 
29.4 
31.9 


+3 .3* 

+4.0** 

+4.5** 


Average Total Earn1nge» Qaartere 2-4 ($]' 


1169. 8B 


1D8S .81 


+100.07 


Average Totat Earnlnge ($] 

Quartar of Random Aeslgnment 
Quarter 2 
Quarter 3 
Quarter 4 


236.22 
3 07.25 
4D5.64 
476.99 


241 .71 
3 07.0 8 
356 .70 
426 .04 


- 5.49 
+ 0.17 
+48.95 
+50.95 


Ever Received Any AFPC PaynentB, 
Quarters 1-4 {%] 


85.2 


86 .2 


-1 .0 


Average Munber of Months Receiving 
AFOe Paynentet Qaartere 1-4 


7.67 


7. 88 


-0.21 


Eve r_ Recel V ed Any AFDC PaymentB l%] 
Quarter of Random AEBlgnment 
Quarter 2 
Quarter 3 
Quarter 4 


B2 .0 
75.7 
65.5 
50.0 


82 .6 
76.4 
67.5 
6 0.0 


-0 .5 
-0.6 
-2.1 
-1 .0 


Average Totat AFDC Paysiente 
Received! Quartere 1-4 ($j 


1974.71 


2085.71 


-111 .DO** 


Average AFDC Peynente Received ($] 
Quartar of Random Aeelonment 
Quarter 2 
Quartar 3 
Quarter 4 


558.10 
536 .15 
460.08 
420.40 


572.49 
566.86 
487.52 
446 .74 


-14.40 
-32.81 
-37. 44** 
-26.35* 


Sample Size 


1664 


823 





SOURCE AND NOTES: See Table 4.2. 
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TABLE 4iB 



VIRGINIA 

RURAL APPLICANTS AN 0 RECIPIENTSs IMPACTS OF THE 
ERPLOYHENT SERVICES PROGRAM 
(AUGUST 1983 - SEPTEMBER 1 9B4 IMPACT SAMPLE) 



PM^OAB (shd Follow -Up Period 


Expe rlnentate 


Co n t r 0 I B 


Difference 


Ever Enplbyedv auertere 2-4 1%]^ 


32.9 


31 .5 


+1 .4 


Average Nuiber of Quartera With 

. . ^ _ ..a 

Employnentp Quartara 2-4 


0,71 


0.71 


-0.00 


Ever Employed (X) 

Quarter of Random Aaaignmant 
Quarter 2 
Quarter 3 
Quarter 4 


19.7 
20.4 
23 .4 
26 .9 


16.7 

fi2 .9 
22 .5 
25 .3 


+ 1 .0 
-2.5 
+ 0.9 
+ 1 .6 










Averajje Total Earhlhgay Quartera 2-4 ($]" 


650 .25 


656.35 


- 8i10 


Average Total Earhlnge ($] 

Quarter of Random Aealgnment 
Quarter 2 
Quarter 3 
Quarter 4 


167 .26 
200.36 
266 .30 
363.60 


156 .20 
202.59 
310.23 
345 .53 


+ 1 1 ;0 8 
- 2.23 
-23. 
i-18.Q6 


Ever Recei vad Any AFDC Peyment Bp 
QuarterB 1-4 [%] 


66. a 


66 .0 


+2 .6 


AvBrage Number bf.MbhthB Rcicelvlng 
AFOC PaymehtBv QuorterB 1 - 4 


6.09 


7 .85 


+0.14 


EvBT Received Any AFDC PaymehtB [%] 
Quarter of Random ABslgnment 
Quarter 2 
Quarter 3 
Quarter 4 


65 .3 
76.5 
67.6 
62.4 


64 i5 
76.3 
67.1 
56.6 


+ 0.7 
+£.2 
+ 0.5 

+3ia 


Average Totel AFDC Paynente 
Rficelvadv Querteri 1-4 ($] 


1733.31 


1711 .66 


+21 .65 


AvaragB AFDC Paymente Racelvad ($] 
Quarter of Rendon Aaelgnmant 
Querter 2 
Querter 3 
Quarter 4 


461 .94 
471 .91 
40 4.11 

375.35 


476.15 
464.08 
404.44 

366.98 


+ 5.78 
+ 7.83 
- d;33 
+ 6.37 


Sample Size 


454 


221 





SOURCE ANO NOTES: See Table 4.2. 
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T^LE 4.9 



VIRGINIA 



Umm mo RORAt APPtlDWlSj impacts of the EHPLOVJCNT services ffiOGRAM 
{AU3UST 1983 - SEPTEMBER 1984 IMFWCT SAMPtE) 







Urban AppLlcanta 


Rural ApplicantE 


jOutcofiis and FoUdw-Up Quarter 


Expari mantel a Control a 


biff^erinca _ _ 


Expa^imenteta Control a 


Dif feranc 


EVBr EiDDLfiVfid. Quaptapfi 

2 - 4 t%) 


62.1 


67.1 


-h4.9 




*>2 ml 


+8.1 


Av ere H un be r of Qua r te r e 
With iBptoyaentf jOtaarters 
8-4 


1.38 


1.22 


+0.15* 


1.13 


0i86 


t«.17 


Evir BiipL^ed IXJ 

Quarter of Rrndon Aeelonment 

Qunrtar 3 
Quarter 4 


48.1 
43.8 
45.7 
48.0 


49.8 
41.6 
39.3 
41.3 


-0.7 
•^2.2 

46.8** 


33.9 
32.6 
37.4 
42.6 


36.0 
29.6 
32.9 
33.3 


-2.2 
+3.0 
+4.5 
+9.3 


AvereQe Totet .Earn^ngBr 
Qusrtera 8-4 ($] 


1712.18 


1565.79 


+146.39 


1347.14 


1^62.28 


+84.86 


Averasei^TotaL Earninga (ij 
Quarter af Random Asslgnrheht 
Quarter 8 
I^Mirter 3 
Quarter 4 


416.82 
4S3.5B 
593.53 


473.69 
496.16 
495.69 
- 573.94 


- 56.86 

- 42.56 
+ 97.84 
+ 91.13 


312.53 
315.33 
450.65 
581 .16 


316.37 
281 .50 
47b.04 
502.74 


+33.83 
-27. 39 

+76.42__ 


Ever Revived Any AFDC 

.ayilWIilbei lAUBrbBrs 1 H I *J 


67.4 


68.7 


-1.3 


7rs.6 


71.9 


+3.9 


Averaoe NoQber cf Months | 
Receiving AFDC Paymentay 
Quarter a 1-4 


4.90 


5.04 


.J o 


5.14 


5;4l 


-0.27 


Ever RecalVBd Any AFOC 
Payment l%] 

I^MBrtar of Random Aaalgnmant 

Quarter 2 

QcBfter 3 

Quarter 4 


60.1 
57.6 
44.6 
38.0 


5t>.8 
56.2 
46.0 
39.9 


+0.4 
+1.4 
••1.4 
-1 .9 


67.6 
?£.0 

_ __ . __. 


67.7 
5P.6 

rJl^Q- 


-0.1 
+6.3 
-1.0 
-5.1 


Average Total AR)C Payments 
Received! Qoartere 1 - 4 ($} 


1234.11 


1335.37 


-101.26 ' 


1tj7L;.t';r 


1081 .32 


- 5.31 


Average AFDC Paymente 
Received t^j 

Quarter of Randooi Aaelghment 

Quarter ^ 

f^MBrtar 3 

Quarter 4 


319.32 
376.63 
286.48 

_2S1 i68_ 


324.74 
403. 70 
323,97 
2£^,96 


• 

-5.42 I 

- 27.07 

- S7.49 

- 31.28 


299.47 
340 ,PS 
231.3-: 
S05.01 


262.48 
321.00 
243.54 
2?0c31 


+16.99 
+18.23 
-18.23 
-23.29 


Sanpte Size | 


669 


341 




_ _ m 


87 




SOURCES MDRC catcutatloha fron CdmindrareeL th of Virginia L 
racdrda frcn the Vi rginia Autonated Client Inf omati on Syatc 


Jnonployroent *> 
]»» and Fai rf 




* >i;-: -acord/^: 


.^-fare 



NQTSt thaae data^ inctude zero vaLuaa for aoDple aoibara not amplsyed cr 2 for i^a nr' 

racelv Ing welfare. These data ere reLreaal orraty uated uaing ordinary leact aqu^ie?^ .vnti f-j: 

pra-rahdan aaai grment charpcteri Btlca of aample nembera. Due to rounding, Vi* r^^. ^ c;: be r^c i T:' 
In calculating anna and differences. 
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T^SLE 4 J cbritinuedi 

•ntered theli,nti'"f «r 2nr^«^i.",^H^"'^"P ''^5^!-*° I)^ calender quarter duHBo thi ^ an "indivlduBt 

-! . L^f ^""^ Miptoyaent ana earnlnoB data. For AFDC paymenta, the quarter of rBnaoih aaelahnieht 
rafars to the Oiree lonoie baglnnlng with the oonGi Fandcn aaelQrrient. "elennant 

ippllcants^tSrSaTrScfplaler '"^""P'" urhan appL i cants (b] rural eppLicants (cj urban 

a- _ _ _ _ _ _ 

nihHni^ of random eselomentp way contai h iocne earnings fron the period prior to 

random a«8ign«ant and la tharafore axctudad fro« laaauraa of total foLLo^? for anpLoymant and aarnln^^^^^ 

im^mtn .n* 4r.^T^V'^^ t^teat was applied to BxpaHnentaL-controL differences. StetlatlwL almiflcance 
leveta are Indicated aai • = 10 percent; e 5 percent; = 1 parent. «itfniTi«ince 

Diffarenoea between erea lepacta nere not algniflcant at the 10 percent level using a tw^-talted 



^teat 
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IfiBLB 4i10 



VIRGINIA 



Umm mt) RUR/JL recipients: IMB^CiS GF lHi EMEtdyH©<T-SERVICB mOSRAM 
(AUGUST 1983 - SEPTEMBER 1984 IMPACT SAMPLEJ 





Urban__RBciptenta 




Rural Raclplehta 


but com B and FotLf-tJp Q UB t ta r 


ExperloehteL a 


Control a 


Olfference 


Experlmentel a ControLa 


Diffetant 


Ever Bsployedr Qi^rtare 

2-4 


36.6 


33.1 


+3.4 




CO .D 


-2.4 


Average Nuaber of QiiartBrc 
With gnpLoynenti Quartari 
2-4 


0.75 


0.65 


+0.1 G* 


0.42 


0.53 


-0.11 


Ever &ptoyad-(X] 

Quarter of Rendbm Aaaionmant 
Qifarter 2 
Quarter 3 
Cuarter 4 * 


16.0 
21.6 

25 ii 
26.7 


12.4 

1:^^.3 

22.4 
25.2 


+3.5* 
+4.3** 

+2.7 
+3.5 


10.0 
12.1 
13.8 
15.9 


7ii 

17.8 
15.2 
19.9 


+2.9 
-5.7 

_ _ -3.9 


Avijfoyf. Total Earh^ngir 
ft'jartara £ - 4 {%] 


845.20 


751 .85 


+94.35 


499.55 


595.13 


-95.59 


Avnragh Tntfil Earni pqe (?) 

Quertei £ 
uuarter o 
Quarter 4 


114.75 
209.41 
283.36 
353.43 


_ OC 9 f. dp? 

176.22 
255.08 
320.55 


*39 -m 

Ti3 C . U _l 

+33.19 
+28.26 
+32.67 


61.18 
121.78 
166.92 
2Q6.84 


59.06 
146.02 
203.69 
245.42 


+ 2.13 
-24.24 
-34.78 
-ai6.56 


Ever Received Any AFOC 
PaymBhtBr Qiartera 1 - 4 (%] 


97.2 


88.3 


- 1-1 


97.5 


96.1 


+1.4 


Ay ir B5i _N be r of Hon th a 
Receiv1ng_AF=DC PayniBntBr 
Quarters 1-4 


9,52 


S RP 
o .oc 




10.06 


9.78 


+0 .30 


^Vjer RBcelvad Any AR)C 
Peyment (xL 

Qoerter of Random ABBlghiheht 

Q LM r tA P 2 

Qi«rtar 3 
fiuartar 4 


96.8 
87.9 

79.5 
73.1 


98.1 

90.2 

82i3 
73^7 


- 1.3 

- 2.3 

- 2,8 

- 0 .6 


97.5 
89^5 
82.9 
79.6 


96.1 
^.3 
62.5 
71.1 


+1.4 
-0.9 
+0.3 
+8.5* 








Average TdteL AFOC Payment a 
Recelvedi Quertara 1 - 4 ($] 


2469.10 


2603.40 


-134.30** 


2189.29 


2137.92 


+51.37 


Average AB)C Paymahta 
Received (9] 

Quarter of Random Aaalgnmant 

Quarter 2 

Quarter 3 

Quarter 4 


718.13 
642.62 
575.68 
532.67 


742.17 
683.06 
617.81 
560.35 


- 24i04* 

- 40 .45** 

- 42.13** 

- 27.68 


609.26 
563.20 
524.44 
492.39 


605.15 
560.51 
506.59 

46a^67_ 


+ 4.11 

+ 2. 69 
+15,65 

+2B*25 


GonpLe Size 


1015 


4Sg 




266 


134 











__ SOJRIX: HDRC caLcuytlpna from CqmcnpniBBL th of Vt rgl n1 a Unampt o^^ Ineuranca earnlnge racordBi 
velfare records from thaVlrglnle Autanated Client Infornetlan Systemi end Fil rf ex Cdiihty AFDC caae fllea. 

_ '^^^^r Z**"^® InP^-y^^.-i^Pro yalurvB for Biople BamberB not onploj'Bdand for earn pi a mem bare oot 

raceiyjng «eLfBrB. Thaae deta are ragreaelorr-actjuatBd oalng ordinary leeat aquaraSp ^ntrottlng for 
pre-randcnt aaaigrinant characterl atl ca of eonple membera. Due to roundlngp there may be aome diacrepancles 
In calculating auna and differences. 

(continued} 
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T^LE 4i10 tcontinoBdi 



?«cau6© of uhderropbrtlng or 1 mine d1 ate termination of wetfare benafltsr the peroentaoe of 
recipients recelvlnfl AFDC paynentG in the quarter of random assljsnmeht does hot equal 10Q. 

9"?''^^'' of ''"'"^om aefigraent the calender quarter durl ng whl ch an Indlvlduel 

entprBd_t^B_8ample for m aarnljiga data. For AFDC peynentSr the quarter of random essl srvnent 

refere to tho three norths beginning with the nonth of rendon aeslgnment. 

RegrBBiri ohs were run on Beperate aubBamplBS of (a] urban appllcante [bj rural applicants 
(c) urban reel plants (dj rural reclplente. 

e^ i 

Uuartar 1r the quarter of random asslghmehtf »ay oihtal n eorae aarnl hgs f rcRn the perl od pri or to 
random assignment and Is therefore excluded from meesuree of total foUow-njp for onpt cement end aernlngs. 

_ ? tHO-talled t-tast wes appllod.to cxperlmantat-controt differances. Statistical elgnlflcance 

levels ere Indl rated as: * = 10 perTOnt; = 5 percent; = 1 percent. 

Differences bet^^een eree Impacts were not significant et the 10 percent level using e 
iMo-taUad t-teet. 
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become 6.H and 6;8 percentage points higher for urban jxperlmentais than 
for urban controls. there were, however, no other statistically 
significant impactsi 

Table UiiQ shows the impacts for urban and rural sample members who 
were recipients at random assignment. Of the total of 1,897 recipients, 
only i|dd came from the rural agencies. Given this small sample, none of 
the differences between rural experimentals and controls were statistically 
signlficaht, with the exception of a significant increase in the proportion 
of jexpertmentais receiving welfare in the fourth quarter. 

For urban recipient experimentals, the story was somewhat different, 
but not as positive as for urban applicantSi There were early^, significant 
employment impacts of 3.5 and ^.3 percentage points in the first and second 
quarters. These dipped in the third quarter, but the gain came back in the 
fourth quarter, although it was not statistically significant at, that 
point. Small earnings gains were als:^ not significant. 

Welfnre savings were more pronounced for the urban recipients. 
Because of the program, urban recipient experimentals received $13^ -SO less 
in welfare payments over 12 months than their urban control counterparts. 
The impacts were for the most part statistically significant and 
concentrated mostly in the second and third quarters. 

Finally^ it is interesting to look at the absolute levels of employ^ 
tneht and welfare payments to see Lome of the consequences of payment 
standards and labor market conditions in the two different areas. For 
example, 57.1 percent of urban applicant controls but only 42 percent of 
rural applicant controls worked at some point during quarters two through 
four. On average, urban applicants received considerabxy more in welfare 
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payments over the course of a year than rural applicants, 9 

VII, Oth e r Subgr oup^-IfflPa^& 

Tables 4.11 and 4.12^ as well as Figures 4.5 through 4.8, show impacts 
derived by subdividing the full sample according to certain characteristics 
usually denoting level of employablllty : prior employment, length of prior 
welfare receipt, educational level and number of children. Behavior of the 
control group shows that, those with fewer barriers to employment (juch as 
those with recent prior work experience) did better In the labor market. 
Those with a high school diploma had higher employment rates than those 
without. However, the Impacts, particularly for applicants, reveal how 
these relationships could change, given some crogram assistance. In this 
study, those groups considered less employable generally had more favorable 
impacts froir program services than those more employable, as discussed 
below. For example, those without high school degrees made bigger gains 
than those with diplomas. It should be noted, however, that, for both 
applicants and recipients, absolute levels of employment were always higher 
for the more employable persons, even though the harder-to-employ 
individuals sometimes gained more from the program. 
A. Prior Work Experience 

The program had relatively large and statistically significant 
employment Impacts for applicants without prior employment, but not for 
those with prior employment (or for recipients in either group). The 
advantage for applicant experimehtais without prior employment over appli- 
cant controls in that situation was 8.2 percentage points, an impact that 
was statistically significant. Welfare savings for the group without prior 
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VXBGINIA 



AFOC APKICANlS: IhMCTS Q^IHE EMEtOYMeiT SEWICB FflOGRAM, 
BY Sa.ECTEp SUBGRaure_ 
(AUGUST 19ffi - SEPTEfeER 19B4 IMFACT SAMH.EJ 





Percent 
of 

SonpLe 


Employ ed Ourl hg 
Quertere 2-4 [%] 


Averege AR)C F^ymenta Beceivad 
bur1n(ji QuBPt^r? ($J 


Oiaracterl etl c 


_Exparlmflntal B^Contr^UB^^I f f erenca 


Experiment/a 


l6 Controls 


Difference 


EnpLoyed During Year 
Prior to R|ndoni 
Aael gnment 

Yas 

No 


57 -g 
42.8 


70.5 
44.7 


67.3 
36.5 


^.2 
+8.2* 


1188,95 

1C20.91 


1369.13 


-148.22 


High School OlpLama^ 
Taa 
No 


51 iO 
49.0 


65.7 
53.0 


61.0 
46.9 


44.6 
^.1 


1133.97 
1267.47 


^170.1/ 
'438&r34 


- 30.20 
-1£1.B8 


Length of Prior Net fere 

History 

Never on AFDC 
Tno Yeare or Leas 
Rbra Than Tcio Year a 


31.5 
4£.2 


58.3 
62.7 
57.9 


58. 2 
52.3 
52.5 


+ 0.2 

+10.4** 

+ 5.4 


934.75 
41 7. 30 


9;8.55 
1208.25 
15Q''.47 


•-44. 80 
-61.21 
-83.66 


Nunber of Onh Children^ 
Ona 

Hbra Than One 


49.7 
50.3 


58.4 
60.5 


54.5 
.6 


+3.9 
+€•9** 


1029.19 
1380.63 


1^38.11 
1332.51 


-178.93** 
+ 48.32 


Age at Randooi Assignment 
Younger ^ah 35 
35 or Older 


60.5 
39.5 


62.6 
53.4 


57.6 
4Z.5 


+5.0 
+5.9 


: 21 2.43 
- 1180.13 


1324.91 
1201.27 


-112.38 
- 21.14 


Scmpte Size _ _ 


ISBS 


857 


429 




857 


428 





SOURCE: MDRC celculatlona from Canraomealth of VI rgl hi s Unarnpl^meht Ihsurence earnings re cords i 
welfare racards fron the Virginia Autdmated Client Information Systenii and Fairfax County AFDC caee fllea. 

NOTESx Thiae datB includaizerp yalgaafor aWRple members not onployed end for sanple mernbers hot 
receiving welfare. These data are ragraaal on-aijj Qstad usihg drdlhary iMat sqi« rear control U for 
pre-rahdom aaalgnneht characterl atl ca of earaple BBmbera. One aet of treaime nt-aub group interectl one nas 

'^^BC^'fieldn at B time, rather^ all slmultaneouslyr Oue to rounding, there niey be some 
diacrepenclea In calculating suns and differences, 
a 

Peraohs were cdhaldered employed during the year prior to random essigrenent if ttiey hed UI 
earnings for any of the four pH or querters. 



The *'yes*' cetegory Includes individuals w1t^ a Geherel B^ulvelency Olplcme. 
c _ 1 _ _ __ :: -_ : j 

01 f ferehtl ale f or fwUy alze were calcuLetBd frq^ termi Eati lates era 

oho*" only f or pne_ or_mpfB_thRn one <^jldi__A few Individuals reporting zero for notiber of children ware 
cleaalfled In the more than ona child category. 



A twD-telted t-teat was applied to l^mpacta within each aubgroup^ c^ Statistical 
algnlf «canca Levele are Indicated UBi * = 10 percent; ** 5 percent; *** = 1 percent. 

Olfferencas ba^aen subgroup 1m pa eta were not significant at the 10 percent level using s 
two-tailed t-taat. 
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TfiBLB 4.12 



VIRGINIA 



AR)C REdPieJ-JB: IMPftCTB OF THE EMPL0YMB4T SERVICE mOBRAM, 
BY saECtED SUBGROUPS 
rAUBUST 1983 - SEPTEHBER 1984 IMRACT SAMPLE] 





Percent 

of_ 

.Sample 




Employed Ouri hj 
Quartera 2-4 


h 


Average AFDC Paymenta Received 
aflrlnfl_auB-rter6 1-4 l$] 


Oiarecterl etl c 


Exparlmaatat^ Control a Difference 


Experinentel 


a Control a 


Difference 


Enptoyed Ourlhg Yeer 
Pri or to Rgndom 
Aeei gnment 

Ye 6 

No 


22.9 
77.1 


64.7 
24.0 


22.6 


+1.4 


2063.72 
2510.41 


2248.47 
2564.90 


-164.75* 
- 74.49 


High School OipLone^ 


38.7 
61.3 


40.3 
29.0 


41.6 
24.5 


-1 .3 
44.5 


2395.34 
2414.32 


2329.2)9 
2624.67 


+ 66.05 
-210.55*** 


L^"9th of Prior WeLfere 
Hi fit orv 
















Never on AFDC 
Imo Yeere or Lees 
Hbre Then Tno Year 6 


-2.6 
25.7 
71.7 


25.5 
40.8 
31.0 


34.9 
39.7 
27.9 


-9.3*** 

+1 .i 

+3.0 


1816.20 
2161.12 
2517. 3£ 


2243 .52 
2140.73 
2650.14 


-427.32 
+ 20.39 
-132.82** 


-- - g 

Nonber of Ovn Children 
















One 

Hbre Then One 


57.9 
42.1 


33.8 
32.9 


32.0 
30.7 


+1 .9 
+2.2 


1^7.07 
2823.35 


2370.52 
2621 .94 


-543.46*** 
+201 .42*** 


Age at Randoa Asel gnnent 
Younger than 35 
35 or Older 


55.6 
44.4 


37.1 
27.3 


35.4 

24,5 


+1.8 
+2.9 


2397.60 
2422.91 


2506.83 
2513.10 


-1 09.03 
- 90.19 


Sampta Si7e 


1897 


12B1 


616 




12B1 


616 





SdtjRCE: MIMC catcutationa froa CowmonweBl tii of Vi rgi hi e Unamployment Inauranca earninge rBCor«te» welfere 
recorda from the Virgihie Automated Client Infomet -^^n Syetemi end Fei rfex County AFDC case files. 

NOTES: Thiae detaJncLude zerp veluee for aampta m^bera not antptoyed and for aaniple nianbere not 

receiving welfare. These date ere regreaalDrt-acU uated uaihg ordinary leaat iquaree» pontrpL Ling for 
pra-rsndom aeai gnment character iati ca of a3intJla_membera.__Dne G!?t of traetroent-aubgronp i nteracti ona was 
•"torad into the regreaaion at a timer rather than att almnt tantLjoaty. Due to rouhding» there roy be eome 
di acrepanci ee in calculating suna and differencea^ 

Bocanaa of w^J^Brre porting or immediet of 
recipiente receiving AR3C paymenta in the quartar of rendom eaaigntnent does not equal 100. 

a i_i _L_ 

RBreonawere considered amployed during the yeer prior to rendom eaaigrvnent if they hed UI 
earnings for any of the four prior quartera. 

b __ 

The *Vo6*' category includea individuate Kith a Garsral Bqaivalancy Diplane; 

c - 

''^^^^o'^BhtieLa for fiswily aize were calculated from a linear interacti_pn term. Eatimatia are 
°"^y for bra or more then one child. A few individuela reporting zero for nunber of children were 
claaalfled In the more than one child category. 

A twff-talted t-tast was appMed to Impacta within each aubgroup category. Statieticel 
•igniflcahce levele are indicated aa: * = 10 percent; = 5 percent; •*• = 1 percent. 

- Differencea between aubgroup impacts were not bignificent et the 10 percent level ueing e 

two-tci t ed t-tBf?t. 
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FIGURE 4;5 



VIRGINIA 

m APPLICANTS AND ^EGlPIENfe: lULATIVE 
EHPLOYHENT PATE, BT PRIOR YEAR EMPLOYMENT STATUS 
(AUGOST 1983 - SEPTEMBER 1984 IMPACT SAMPLE) 
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FIGURE 4.6 
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AFDC AP.PLIC/SNTS AND REGIPIENTSi GUMULATIVE 
EMPLOyMENT miB. BY HIGH SCHOOL DIPLOMA STATUS 
(AUGUST 1983 - SEPTEMBER 1984 IMPACT SAMPLE) 
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FIGURE 4.8 
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m APPLICANTS AND RECIPIENTS: GUhut.ATIVE 
AFD9 PAYMENTS, B? HIGH SCHOOL DiPLOMA STATUS 
(AOCyST 1983 - SEPTEMBER 1984 IMPACT S/^.MPLE) 
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f^iiplwyment came to $11|8 compared to only $20 for their more employable 
counterparts. Tllese savings, however, were not statistically significant. 
(See Figures li.5 and k.6.) 

B. Prior Welfare DependenGV 

Program impacts were stronger for applicants who had some prior 
welfare receipt. Among applicants who had previously had their own case, 
the welfare savings ranged from $61 to $84. While these amounts were not 
significant, they were somewhat larger than the impacts for applicants who 
had not b^en bh welfare before ($i*5). Time on welfare also influenced 
employment impacts, which were higher for those on welfare previously. At 
a statistically significant level of 10. i| percentage points, the employment 
Impact for applicants receiving welfare for two years or less far exceeded 
that for applicants who had never had their own case (0.2 percentage 
points) . 

C. Education Level 

Among controls, a high school diploma or equivalency certificate was 
associated with greater subsequent employment and lov^cr welfare payments. 
However, program-induced employment and welfare impacts were concentrated 
instead among both applicants and recipients without high school degrees. 
As Indicated in Table i|.12, recipient experimentais without such degrees 
recei.ci lower welfare payments than controls, a $211 amount that was 
statistically significant compared to an increase in average welfare 
payments of $66 for those with diplomas. While payment impacts were not 
significant for applicant Experimentais (eee Table 4.11), the welfare 
savings of $122 among those without diplomas were four times larger than 
the savings from those with diplomas. (See Figures 4.7 and 4.8.) 
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A large family Is usually seen as a potential eopioymeht barrier^ but 
both applicant and recipient subgroups overcame this barrier as a result of 
prdgraiD services. Among applicants, the employment impact was a statistic- 
ally significant 6.9 percentage points {ccnnpared to a nonsignf icant 3.9 
points for women with only one child). Among recipients^ impacts were not 
statistically significant for either large or small families, but the 
emplbymeht gain was slightly larger for recipients with more than one 
child. 

In contrast, welfare payment reductions seemed to be concentrated 
among both applicants and recipients with one child. For recipients, the 
reductions were statistically significant at $5^3, a level 23 percent down 
from the control mean of $2,371. For applicants with one child, the 
reductions were ai30 significant; at $179, average payments were down 15 
percent from the control group average of $1,208. For both applicants and 
recipients with more than one child, impacts were in the direction of 
somewhat greater welfare receipt for experimentals. 

VIII. Impacts During the Base Period for Pr^ections 

The main focus of this chapter has been the program's impacts for the 
full impact sample over four quarters after random assignment, as well as 
the impacts for the early enrollees in the full sample, followed over six 
follow-up quarters; Since the six-quarter impacts consistently carry on 
the stox^ begun in the short term, it is plausible to expect that the 
program impacts found at the end of this follow-up may continue or even 
increase beyond that point. These future progrr are important, 
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because they are used to compare t program's benefits to its costs over 
time the purpose of the i allowing chapter^ A longer-term focus of f*ive 
years is used and considered hieciessary sihcis most costs of the program are 
incurred in the shbrt-ruh| When experimentals are still active in the 
prograiD, while benefits may accrue over a longer period of time as people 
continue to work and reduce their use of welfare and other transfer 
programs. 

As will be seen in Chapter 5, future program effects are estimated by 
extending, or extrapolating, the impacts observed during the last two 
quarters of follow-up, using a number cf assumptions about the direction 
and rate at which these impacts will change. The last two fbllow-up 
quarters are used as the base period for these projections because, in 
comparison to earlier impacts, they are more likely to provide a more 
accurate indication of future patterns. 

For all sample members, the projection base period covers the months 
of January through June 1985 for earnings data, and April through September 
1985 for AFDC welfare payments data. Since sample members entered the 
study on different dates, these base periods represent different lengths of 
foiibw-up relative to a person's date of random assignment. For example, 
for siampie members randomly assigned during August ' September I983, the 
base period for earnings is their seventh and eight.: follow-up quarters. 
For those randomly assigned during July through September 1984, the base 
period is their third and fourth foilbw-up quai-ters. 

Table i|.l3 summarizes the impacts during the base period. Although 
these impacts are not statistically significant, there is a fairly 
consistent pattern of earnings gains and welfare payment reductions. The 
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TABLE 4.13 



ViflGlNIA 

ALL AFDCi IMPACTS ON EARNINGS AND AFDC PAYMENTS 
IN PROJECTION BASE period", BY PERIOD OF RANDOM ASSIGNMENT 
(AUGUST 1983 - SEPTEMBER 1984 IMPACT SAMPLEj 





1 




Period of Random 




Ab6I gnment 







OutcbiiieB for Last Two 
Follow-up Quarters 


August - 
September" 
1983 


October - 
Deceiiiber 
1883 


January - 
Morcti 
1984 


April - 
June 
1984 


July - 
September 
1984 


Full 
Sampl e 


Average Earningp ($) 


148.16 


199,25* 


197.32 


87.70 


119.85 


165.57 


Average AFDC 














Payments (t] 


-84.74 


-29.10 


-36.90 


-41 .46 


9.95 


-35.07 


Sanpla Size 


377 


111B 


806 


5QB 


373 


3182 



J^"^ catcutBtlans from Commonwealth of Virginia welfare una Unemployment IneurehcB 
earnings rocordsi 

NOTES; 'projection Beee Period refere to the lest two quarters or AFDC peymentB end Unempl byment 
Insurance earnings data evellabte for each Individual. Thle rofers to Querters 7 end 8 for Bernlnge 
and fl and 9 for AFDC peymente for thoES randomly acelgned between August and'September 1983; OuBrtere 
B end 7 for eernlngs and 7 and B for AFDC paymahts for thoee randomly c.signed between October end 
December 1983; Ouartere 5 end 6 for eernlngs end 6 end 7 for AFDC payments for thoae randomly eeelgnad 
between Januery end Herch 1984; Qoartere 4 end 5 for earnlngG end 5 end 6 for AFDC peymente for those 
randomly eeBlgnad between April e^d June 1984; ahd Duertere 3 end 4 for Bernlnge end 4 end 5 for AFDC 
payments for those rendomly aeBlgnBcl between July end September 1 984. 

These data Include zero veloes for eeniple niembers not employed end for eample members not 
receiving welfere. These dete ere regreseion-sdj usted using ordinery least equeres, controUIng for 
pre-randoffl essignment cheracteristlcs of sempte mambers, 

A two-tailed t-test was applied to differences bstween experlmentel end control groupe. 

Statistical elgnificence levele are Indicated as: ♦ = 10 percent; = 5 percent; = i percent, 
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estimates of future effects computed In Chapter 5 v^ill be most strongly 
influenced by the base period effects for persons randomly assigne .^ing 
(Dctbber to December I983 and January to March 198^*, since these groups 
together include 60 percent of the research sample. For these two groups, 
earningsi gains were close to $200 during the base period, ^shile welfare 
payments were reduced by $29 to $37 respectively, (See Appendix Tables F.5 
and F,6 for base period impacts for applicants and recipients separately <. ) 
Chapter 5 will discuss in detail the assumptions used aborit the 
continuation of future effects so that, for each sample member, a total of 
five years of data exists^ consisting both of data collected during the 
jstudy period and estimates of future benefits, using base period impacts^ 



-124- 



179 



CHAPTER 5 
BENEFIT-COST ANALYSIS 



I. Introduction and Summary of findings 

This chapter weighs the economic benefits and costs of ESP to present 
an overall assessment of the value of th^ program. It draws on the find- 
ings of previous chapters and also utilizes data gathered specifically for 
the benefit-cost analysis. Applying techniques developed in other evaluf:- 
tions of social programs,^ the benefit-cost i'.nalysis asse-^ses ESP from 
several distinct viewpoints, particularly that of the i5eople toward whom it 
was directea — rlFDC applicants and recipients >- and of the government 
budget. Thus, within the constraints of the available data, the anaiysir^ 
examiijes whether or not the program achieved its central objectives of 
improving the economic -^.anding of people dependent oh welfare and reducing 
the burden of welfare o.i the government. The analysis also considers the 
effects of the program on society in general and oh taxpayers^ (As 
explairiOv? later, the effect on taxpayers is similar but not identical to 
that on the government budget.) 

This analysis tatimates benefits and costs over a five-year period; 
although most costs were incurred when enrollees were still in the program, 
benefits may accrue over a longer time as people formerly depehdeht oh 
welfare continue to work and pay taxesi Therefore, the analysis estimates 
effects and costs after data collection, using alternative assumptions 
about the way in which effects calculated for the observation period might 
change after it ended. (Section III.E explains the procedure by which 
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effects were extrapolated beyond the observation period;) 

Over the five-year period (including both observed and extrapolated 
estimates)^ ESP enrolleiss, and applicants in particular, were better off as 
a rasult of the program because their increased isarnihgs exceeded their 
increased tax payments as well as their losses in welfare and in payments 
from other transfer programs. The government budget also benefited. Over 
the same five-year period^ the estimated Increase in tax revenues and 
savings from welfare and other programs exceeded the costs of ESP services 
for applicants. For recipients, budget savings and program costs were 
v^bout equal. 

II. \i iB Analytic Approach 

In dett; mining th^ benefitai and costs of ESP, the analysis estimates 
the value of the program's effects on key outcomes, and the cost of the 
resources used in prodv-cing those effects. the main outDome va ables for 
which MDRC collected data include the earnings and AFDC payment: discussed 
in the impact analysis in Chapter ^t, and Unemployment Insurance benefits. 
In addition, the benefit-cost analysis considers a variety of outcomes not 
dir::^ctiy measured but for which values could be imputed: the fringe 
benefits of regular Jobs, tax payments, Medic:iid> Food 5 imps, transfer 
program administrative costs, and the value of output produced by enrollees 
in ESP work experience jobs. The analysis weighs experir*^ntal-control 
differences in these outcomes against costs which include: the expense of 
operating ESP; expenditures for support services such as child care and 
transportation; and the costs incurred by JTPA agencies, public schools and 
community colleges which provided education and training services used by 
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tnembt.'S of the experimental and cb»^* rol groups. 

The analysis exainlni.3 differences between experiinentals and controls 
In order to address the question: What are the average benefits and costs 
of r.SP pei^ experimental above arid beybrid what would have happened In the 
r' ;■ . of the program? it Is important to emphasize that these estimates 
ar . - ^"eraged both over members of the experimental group who participated 
in program activities and those who did not. One risasdri for iriclubing 
nonpartici pants is that ESP's mandatory participation requirement rn^y have 
influenced the behavior of this group — for example, by deterring some 
welfare applicants from pursuing their applications, or by encouraging some 
recipients to leave the rolls to avoid the requirement. Moreover, ome 
costs are associated with nonparticipants, including those of enforcing the 
mandatory participation requirement, as well as of program reporting and 
administration required for thec^ activities. 

The period for which dpta were gathered (referred to as the observa- 
tion period) varies by data source and date of random assignment for each 
sample member. Random assignment began in August i^83 and extended until 
S'-^^^^^mber 198^1 (f?ee Table 2.1), and dat-^ collection for some variables 
r r. d through November 1985. The period of observation ranges from 
quarters for earnings data for the last enrbllees randomly assigned 
tb bver twb and a half years for some of .he participation data for the 
earliest enroiiees. Benefits and costs accruing after the end of the 
observation period (e^gi , June 1985 for earnings data and September 1985 
for AFDG data) up to a point five years frcwa the date of random assignment 
have been estiiaated for each sample member oh the basis of observed data 
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and a series of assumptions. All benefits and costs have been valued in 
19811 dollars, 

Whiie the available data permit estimation of a wide array of benefits 
and costs, some potentially useful ihfbrmatibh was not included In this 
study, such as General Relief payments as well as the earnings or other 
income of family meiabers of individuals in the research sample* Hbrebveri 
as explained more fully later in the chapter^ potential intangible effects 
of ESP, such as changes in participants' self-esteem or in the quality of 
their family life, could not be determined, Interpretatjbns of the 
benefit-cost analysis presented below should recognize the scope of the 
analysis. 

Illi. Economic Value of Program Effects 

A; Increased Earnings and Fringe Benefits 

As seen in Chapter i|, ESP yielded ^ease in the level of earnings 

i f experimentals over the conti " (It is important to underscbrie that 
this calculation is based on all experimentals — those who became employed 
and those who did not.) Table 5i1 shovs these earnings gains during the 
observation period. The experimental-control difference was $250 for the 
full researcn sample, with the gain amonf^ applicants ($3113) exceeding the 
gain among recipients ($199). The table slso presents the valu"^ of fringe 
benefits: $il5 for the full sample, with applicants again receivl ig a higher 
amount thrn recipients ($62 ve-sus $36). (Fringe benefits were estimated 
tb be 18 percent of earnings, a rate hased or national employment compensa- 
tion data for the types of low-wage jobs held by most experimentals d 
ccnt*^ols. )3 
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TAB LE 5 .1 



VIRGINIA 

ESTIMATED EXPERIMENTAL - CONTROL DIFFERENCES IN EARNINGS, 
FRINGE BENEFITS, AND TAXES PER EXPERIMENTAL 
DURING THE OBSERVATION PERIOD, BY WELFARE STATUS 



Component of AnBlycIs 


Full Sampl e 


Apj: L 1 ;:a nt 8 


R ^ 1 pi e n 1 6 


Ea r n1 ngs 


$250 


$34ri 


11 gs 


Fringe Benefits 


45 


62 


36 


Taxes 








Federal Income Tax 


1 5 


1 5 


IB 


State .Cn;:ome Tax 




3 


2 


SorieL Security Tex 


17 


23 


13 


State SeLee and Excise Taxes 


1 


2 


1 


Total Taxes 


35 


43 


32 


SOURCE: MDRC catcuLetlone fr 


om Un em p t oyme n t 


Inearance records 


a rt d from 



publlshad dat« on tax rates end emptoyee fringe benefits; 



NOTES: The results are besed on a sempte of 21 38 e x pe rime n t a t s and 1044 
controls (1285 applicants^ 1 697 reel pi en^.s? * end are e xp r e ssed 1 n 1 984 dotters. 
The differences nre regression- adjusted using ordinary teast squeres, 
cbntro.ling for pre-random aesignment characteristics of sample members. 
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The estimates In Table 5.1 present eariii.. i;.. g:^ . -.a . Inoluoe those 
that occurred beyond the fbllbw-up period common to 3ample membev^s 

that is, beyond the fourth quarter of follow-up Vc^ individuals on whom 
such data were available.^ Cdnsequehtly^ these estimatv^s differ from this 
earnings impacts for the full sample presented in Chcpter which instead 
cover the follow-up shared by all sample members. Data for the entire 
fdlldw-up period were used in the behefit-cdst analysis in drder td 
minimize the length df time over which dbserved effects had td be extra- 
polated to cover a five-year period. 

Careful interpretation of the earnings effects presented in this 
chapter requires attention to the issue of statistical significance. As 
noted in Chapter employment impc -cs were sometimes statistically signi- 
i'ic^nt, particularly near the end of follow-up for applicants and the full 
sample, but the corresponding earnings gains t*ere rot and ,;herefore are 
subject to some uncertainty. (That these earnings gains were not stat- 
istically significant is due in large part to the wide variance typicjal df 
earnings in any population c ) However, the impact analysis did reveal a 
pattern of stable, and in some cases increasing, earnings gains over time, 
suggest*^ ng that the measu?^ed earnings gains were not simply due to chance. 
Increv^sed Tax Payments 
Experimehtais' earnings gains resulted in increased tax payments, 
including federal and state income taxes. Social Security payroll taxes, 
and state s£c9s and excise taxesi Using the relevant tax rates, this 
evaluation imputed these taxes from earnings (total earnings in the case of 
payroll and sales taxes, and earnings over a base amount for income taxes), 
other income (for sales taxes), marital status, and number of dependents. 
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The estimation of federal taxes i^oi uded a deduction for the Earned Income 
Tax Credit, and the computatldh of sales taxes took account of average 
consumption patterns. The results In Table 5.1 show that the total taxes 
paid by experlmentals during the observation period Increased by $35 per 
experimental, or percent of total earnings, ^ 

C. li' c reased Output in Work Experience Jobs 

ESP participahts who took part in the program's work experience compo- 
nent not only gained job experience, but also provided services in various 
clerical, maintenance and food service positions. The benefit-cost 
analysis includes the value of these services since the community-at-large 
benefits from the increase in services^ 

The value of the work performed in these Jobs was estimated by deti^r- 
mlhlhg the compensation that employers would have had to pay for other 
workers to provid.'^ the same services/ The first step in this estir^iation 
was to determine the relativ<f productivity of part5 . ! ;>aht3 compared to that 
of regular workers in comparable jobs. EvaluatibriS by participants* supe - 
visors gathered for the worksite survey provided 9, ;'atib comrar ing the 
output of participahts with that of regular workers.*^ For each participant 
evaluated, this productivity ratio was multiplied by the average hourly 
wage rate (marked up for fringe benefits) of regular workers to yield an 
estimate of the value of the pprticipaht's cxjtput per hour of wbrk.^ The 
average hourly value of output was then multiplied by the average J/r^ngth of 
.^tay in work experience jobs (available from program records) to obtain the 
average value of participants* Mork, 

Supervisors judged that the i:r'6duiit.t'''ity of work experience parti- 
cipaht.s was 38 percery: th.^t of regular wbrk^^.^s in similar types \f jobs. 



As indicated in Chapter 3, ESP ehrbllees who e^^e^ed work experience 
positions spent an average of 19*1 hours in those jobs. Following the 
approach explained above, the average value of output was determined to 
$826 per work experience participant and $10^1 per exporimentai* The v;^' 
of output was considerably higher for recipients ($ii|5) than for applicants 
reflecting differences in the proportl applicant and recipient 

experlmentais assigned to work experience.^ 

Di Reduced Dependence on Transfer Programs 

As shown in Chapter i| , expt rimentais became less dependent on AFDC 
income as a result of ESPi To a lesser extent, the program also affected 
their use of Medicaid, Food Stamps, and Unemployment Insurance, although 
hot always in the same way. 

For the benefit-cost analysis, experimental-cbntroi differences in the 
receipt of AFDC and Uhezrployinent Insurance were estimat :jd from AFDC and UI 
records during the entire dbs^ervatibh period (in contrast to the follow-up 
period common to all sample members used in the analysis of AFDC impacts). 
Differences in other transfer payments were not directly Measured but were 
estimated from a variety of sources. Differences in Food Stamps were 
imputed on the basis of household income (including earnings, AFDC and UI), 
takiiig into cbhsideratioh the earnings disregard as well as medical 
deductions -- all of wIj^ h are used to determine eligibility for Food 
Stampb and the amount of permitted benefits."'^ Differencet. in Medicaid use 
were imputed i*bm observed differences in AFDC receipt and the rules 
covering Hed-«.caid el.lgibility . A person who is on AFDC 1' automatically 
entitled to receive Medicaid, and remains eligible for a Lfied period 

->f time after l&avim the rolls.'*'' Thus, the average change in the value 
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of Medicaid was determined by multiplying the experlmental-cdntrbl 
difference: in the months of statutory Medicaid eligibiiity by th^ average 
monthly -:c.: icaid payments to AFDC recipients in Virginia during igS^tJ^ 

'"i i^e ^.2 presents the results. Overall, the program proltced a net 
rede 3 tier of $;3^ in the average value of transfer payments recelvei by 
experitnentais cOTpared to controls: About 96 percent of this reduction was 
due to the decrease in AFDC payments. ''S To a small extent, losses in AFDC 
were offset by increases in Food Stamps, which rose by $i| per experimental 
for the full sample. 

• The reduction in tranr'i'er payments brought about a small deo, ease ($13 
per experimental) in the overall cost of administering those payments. 
Changes in administrative costs were estimated in several steps. First, 
for AFDC and Medicaid^ the experimental-control differences in months on 
AFDC an<" tobhths eligible for Medicaid were determined from AFDC records, 
•^hese u.^ ferehces were then multiplied by the pertinent average monthly 
^dmiiilstrative cost per AFDC adult recipiency or former recipient in the 
case of Medicaid (since eligibility for Medicaid is not lost imiaediately 
after leaving the welfare rblls).''^ For Food Stamps and UI, the change in 
administrative costs was bbtaihed by multiplyihg the average experimehtal- 
control difference in transfer payments by an estimated average cost per 
dollar of traii^^fer'. 

E. £uture_ Ef fects and Total Results 

The effects discussed thus far pertain only to the period oi observa- 
tion. Yet, program effects may endure beyond that point. In order to 
account for the full effects of the program, it is therefore necessiry to 
project beyond the end of the observation period. 
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TABtE 5i2 



ESTIMATED EXPERIMENTAL - CQNTRet DIFFiRENeSS IN TRANSFER PAYMENTS 
AND ADMINISTRATIVE COSTS PER EXPERIMENTAt 
DURING THE OBSERVATIQN PERIODr BY WELFARE STATUS 



Typa of Payment or Cost 


1 

FuL I Sampt e 


App t 1 CG n t e 


Red pi entB 


Transfer Payments 








AFDC 


$-1 28 


$-1 45 


$-128 


Medl eel a 


- 6 


"21 




Food Staptp3 


4 


11 


- 2 


Unempt oy ment 
Com pe n 8B t1 on 


- 3 


5 


- 9 


Tot it ^TrenBT: : 
Pb \ me nt 8 


-13 4 


-1 50 


-139 


Transfer Admlr^uzurrilve Costs 








AFDC 1 


-13 


-24 


- 5 


Hedl eel d 


- 1 


- 2 




Food Stamps 


1 


2 




Unem pi oy ment 
Com pe n sa 1 1 on 




1 


- 1 


Totel Transfe" 

Adm { n1 s t r e t i V e Costs 


-13 


-23 


- 6 



6DURCE: MDRC catcutatlone from AFDC end Uhemployment Ihsurence eern1n§6 end 
payments reccrde*^ end from pubMshed and unpubMshed date bh adm 1 ni s t r e t1 v e 
costs for AFuO.j Hedlcaldt Food Stamps and UnemptDyment Insurance, 



NOTES: the results ere be sad o n _ a sempt e of 2138 rxpe r 1 mentel s end 1044 
controls (1285 appMcantsi t 897 r e c1 pi e nt s ^ , a n d a re expressed In 1884 dollars. 
Tha differences are regreesl on-adj ustad ae^ ng ordinary teast squares, 
controlling for pre-random assignment characteristics of sample members, 

a -. 

Estimeted value less then $C 50. 
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Using a number of assumptions, the analysis extrapolates outcomes for 
each sample member so that the combination of observed and extrapolated 
valuiBs covers a fiv3-year timespan beginning at the point of random 
assignments Five years was chosen as the timespan since it is approxi- 
mately the average length of time AFDC applicants remain on the rolls 
nationally; therefore, in the absence of ESP, the ^r^^y experimental 
Would leave the welfare rolls in about this " of tii.e. Extrapolation 

over a longer time period than reijuireG for thirj five-year timespan would 
involve more speculation. Because the amount of observed data on sample 
members varies according to the date they entered the research sample, the 
length of extrapolation required to estimate results over five years also 
varies by samnle mem^^er. For example, about two years of earnings data 
(which were collected through June 1985) are available on persons entering 
the sainple August 1983f leaving about three years for extrapolation. In 
contrast, about one year of data were gathered for those entering in July 
198^, requiring extrapolation over four years. 

Extrapolation of program effects requires deciding on the base period 
from which extrapolation is made, making assumptions about the rate at 
which experimental-control differences decay over time, and choosing 
appropriate discount rate. The most recent avail-^ble data oii 
effects were considered most appropriate for predicating future e* t 
Therefore, the base period selected for est.vapolDticn was the last two 
quarters of follow-up available for each sample member. Section VIxI of 
Chapter i| has already shown the experimental-control differences in 
earnings and AFDC payments that occurred during this period. For the 
benefit-cost analysis, program effects daring this base period were also 
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estimated for fringe benefits, tax payments ^ non-AFDC transfers, and 
transfer program administrative costs, using the same procedures used to 
measure or impute values for the observation period.'''^ 

the decay rate is the rate at which the base per:*.od estimate is 
assumed to change over time. Because of this study's rv '.atively short 
foiiow-up, it is difficult to predict long-term trendb from the available 
data. Different assumptions hr.ve therefore been used to compute a riaJige of 
estimates. One assumption is that the magnitude of the experimental- 
control difference observed dr.ring the base period will continue unchanged 
during the extrrpolation period. This assumption is supported by the 
finding that the overall employment and welfare Impacts for the early 
sample followed over the longer tern were sustained or increased in the 
final foiiow-up months. Moreover, similar trends have been observed in 
other studies of employment programs for welfare recipients.''^ Yet, other 
studies suggest that impacts do decline with timco For example, a national 
study of the WIN Program found that earnings effects decayed at a rate of 
22 percent annually for female sample members. On this basis, an annual 
decay rate of 22 percent was applied as an alternative assumption. (A more 
extreme assumption — that there are no futui^e effects is considered in 
Section V of this chapter in assessing the sensitivity of the overall 
resiults to different assumptions.) 

The effect of inflation on the value of future program effects was 
avoided by making all projections in 198^ dollars from a base period that 
falls within that year. However, all extrapolated results were discounted 
to their 198^ dollar values to adjust for the value cf foregone investment. 
(A benefit received later in the follow-up period is worth less than the 
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same benefit received earlier due to the lost opportunity to Investi) A 

real discount rate that is^ a rate adjusted for inflation of 5 

percent per year was used in this analysis, 

table 5-3 presents, for the full research sample, the observed^ extra- 
polated and total estimates of program effects covering the five-year time 
period. (Appendix Tables G.1 and G,2 show estimates for applicants and 
recipients separately.) the column of the table headed Common Follow-up 
shows the effects estimated for the portion of the follow-up period avail- 
able f^r all sample members (quarters two through four for earnings ^ and 
one through four for AFDC payments). Since the impact analysis in Chapter 
i* focuses on this period, these estimates for earnings and AFDC payments 
are the same ones reported in Table The column headed Additional 

Follbw-up provides estimates of program effects during the subsequent 
quarters of follow-up for those sample members who enrolled in the study 
early enough to have more than the follow-up available for all sample 
members. (However, these effects are averaged over all experimentals. ) the 
values in the two columns under the observation period sum to the total 
observed values presented in Tables 5.1 and 5.2. 

The column in Table 5.3 headed Extrapolation Period presents a range 
of values with the first number calculated assuming a 22 percent annual 
decay rate and the second assuming no decrease or increase in prograin 
effects during this period. The final column, which is simply the sum of 
the other columns, indicates the estimated program effects at five years 
after random assignment. Again, the range of results presented reflects 
the alternative decay assumptions. 

As seen in Table 5.3, the extrapolated estimates are much larger than 
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TABLE 5.3 
VIR6 INIA 



EXPERIMENTAL - CONTROL DIFFERENCES IN PROGRAM EFFECTS 
P^R EXPERIMENTAL DURING THE OBSERVATION PERIOD, 
EXTRAPOLATION PERIOD, AND AT FIVE YEARS AFTER RANDOM ASSIGNMENT 

ESTIMATES IFOR THE FULL SAMPLE 



Dot cone Vi r1 abl e 



Ea rn 1 ngs 
Fringe BehefltB 
Tax Peyments 
AFDC Peytnents 



Other Transfer 
Pay men t 8° 



Trahsfer Progr^in 
Ade 1 nl it rati on 



O bee rv a tlon Period 



Combo n 
Foil ow-u p 



Adai t1 bna^ 
Foil ow-u p 



-1. 



$B1 

i 5 
1 1 

19 
- 5 



1169 
30 
84 
-45 

-24 

- e 



Ext rapql etl on 
Perl od^ 



$749 to $1090 

135 to 196 
114 to 166 
-1 16 to -230 

-95 to -135 

-3 7 to -53 



Five Year 
Tbtel 



$999 to $1340 

190 to 2 41 

149 to 201 

-2 90 to -3.59 

-100 to -140 

-5 0 to -6 6 



SOURCE: MDRCceLcutetlohs from Unemployment Insurence eernlnge and paymente 

recordBf AFDC pa y me n t b r eco r d e j e nd p ub U sh ed end unpubUshed de te on tex retes. 
employee fringe beneflte, end edm 1 nl 6tre 1 1 ve coete for AFDC, Medicaid, Food Stamps 
and Uhemploynent Ineuranca, 

NOTES: The raiutCa are beeed on a sample of 213B a x pe r 1 me n t s t s end 1044 

controls M2B5Bppt1cantB, 1 BB7 reel pi ants) , and are expresaed ln 1 994 dottare. The 
dlfferancee are regresslon-adJaBted using ordlhsry li>8Et equfires, controlling for 
pre-random Bsslgnment ch e ract a r 1 s 1 1 cs of sample iDamberB. 

a— _ 

j"*"^" coBBbh fbllow-up period covers querters 2 through 4 for sernings, 

fringe benefits and tex payments, and quartere 1 through 4 for AFDC payments, 
Medlceld, Food Stamps^ Unamploymeht Ihsurahce payments end edm 1 nl st r a t1 ve costs, 

b _ _ : 

"^^^ additional follow-up period covers all the avelleble f bl Ibw-up 

qaartara after quarter 4. The number of additional follow-up quarters depends on 
each aaaple BaBber's date of random aealgnment. 

c _ : _ L _ _ 

reflect alternative assumptlbhs abbut the chenge In Impects 
after dete collecton ended. The f 1 ret number of each range essumes thet progrem 
effects deci Ine by 22 percent per year during the ex t r e po t i t1 o n period! the second 
number a as one b ne decay or Increeee. 



The five year totals ere the sum of the eBtimated effects during the 
obseryation and extrepole ti on periodP, 



6 



Unemployment Insursnce. 



Includes Mediceld, Food Stemps end Unemployment Insurence. 
Includes e dm 1 ni st re ti ve costs for AFDC, Medicaid, Food Stamps and 
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the observation period values for all outcomes and account for a larger 
proportion of the estimated five-year effects. For example, the extra- 
polated estimates for earnings account for 75 percent or more of the 
five-year totals, while the extrapolated values for AFDC payments account 
for MO percent or more of the five-year totals. As will be seen in Section 
V of this chapter, the extrapolated effects are critical to the overall 
benefit-cost results. (Section V also discusses the conclusions reached 
when extrapolated effects are not included in the benefit-cost analysis.) 

Table 5.3 also shows how the program's estimated effects over five 
years vary according to the different assumptions about decay. For 
example, the experimental-control difference in earnings after five years 
is estiinated at $999 assuming a 22 percent annual decay rate, and at $1,3^*0 
assuming no decay or increase. The change in AFDC payments over the same 
period of time ranges from $-290 to $-359. Given the uncertainty about the 
true value of program effects after the observation period, both sets of 
estimates will be used in assessing the net results of ESP in Section V. 

IV. Costs 

Experimental-conti dl differences in costs were estimated for several 
types of expenditures including: ESP operating costs, payments for support 
services such as child care and transpbrtationi and the operating costs of 
education and training providers. These costs were estimated separately 
for applicants and recipients as well as for the sample as a whole. 
Average costs were calculated per experimental as in the case of program 
effects. All estimates exclude expenditures made only for research 
purposes (such as for random assignment and special data collection), since 



these S'^tivltles did not contribute to prbgram impacts, 
A. ESP Operating Cost s 

Prbgram operating costs were derived from a variety of data sources, 
including state and local jSSP and DSS fiscal records (primarily covering 
from January through December 198^1),^'' VACIS, and case file records of 
several subsamples drawn from the main research sample. (See Chapter 2 for 
more detail on these data sources.) in addition, in order to determine the 
cost of program components, interviews with program staff investigated how 
they apportioned their time over the program functions for which they were 
responsible,^^ 

This time study considered six prbgram compbnents: initial assessment, 
reassessment^ individual Job search, group job search, work experience, and 
education and trainings However, staff members couid not deteraine the 
time spent on separate activities precisely. Distinguishing the cost of 
reassessment interviews from the cost of other activities was particularly 
difficult due to the broad range of concerns addressed during reassessment. 
For example, staff would typically use these sessions tb review a parti- 
cipant's experience in individual job search, and to discuss assignments to 
work experience positions or education and training options. Reassessment 
interviews were also used to provide counseling that might not be specific 
to any particular program activity. Thus, the costs reported separately 
for job search, work experience^ and education and training actually refer 
to the costs of these components above and beyond the resources spent on 
them during reassessment interviews. 

The cost of each component was estimated in several steps. The first 
step was to determine the "unit cost," that is, the average cost of provid- 
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Ihg the i5t±v±ty to one individual. 22 (jn calculating thli unit cost, ESP 
enroiiees not in the research sample were cbhsidered. As explained in 
Chapter 2, not all ESP enroiiees on welfare entered the research sample. 
Moreover, some agencies enrolled recipients of General Relief and Food 
Stamps into ESP. ) 

The next step was to multiply the average unit cost of a component by 
the experimental-control difference in the percentage of persons assigned 
to it. (As explained in previous chapteis, a few controls mistakenly 
received ESP services, and a substantial number enrolled in education or 
training programs in the community.) These assignment rates were estimated 
from the same data used in the participation analysis in Chapter 3; 
however, in estimating costs over a five-year timespan, the benefit-cost 
analysis used all follow-up data available for the early enroiiees (that is 
data from beyond the nine-month period common to all sample members used in 
Chapter 3). Moreover, the assignment rates were regression-adjusted 
experimental-control differences, in part to assure consistency with the 
appro^iCh to estimating program effects.«^^ 

This evaluation understates the true costs of serving experimentals to 
the extent that the estimated assignment rates do not capture assignment of 
individuals who entered program activities after data collection ended. 
These assignment rates are more likely to have been underestimated for 
later enroiiees since they had a shorter follow-up period than earlier 
enroiiees. (See Figure 3.2 In Chapter 3.) As a partial correction for 
this problem, assignment rates for later enroiiees were extrapolated based 
on the experiences of the earlier group. However, since most individuals 
who became active in the program did so early in their program tenure, the 
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extrapolated results increased the estimated participation rates only 
slightly. 

Table 5.fl presents the estimatisis of ESP operating costs. The estiiated 
overall cost per experimental, which is the sum of the cost of each 
component, is $388 for the full sample. The higher estimate for recipients 
($^79) than applicants ($251) reflects the fact that a greater proportion 
of recipients than applicants participated in each program component. 
Assessment and reassessment functions together account for over half of 
these operating costs. Individual and group job search activities that 
took place outside of reassessment interviews together comprise about a 
fifth of operating expenses, and work experience activities amount to 
soEQewhat less than a fifth. About 5 percent was spent on referring 
participants to education and training programs in the community, for the 
most part to pay for the time ESP staff spent assigning participants to 
educational and training services, and monitoring their progress. (Only 
the Chesapeake agency, which operated an on-site GED program, provided 
education or training itself.) 
Bi Support Services 

To enhance opportunities to take part in ESP activities bi to ease the 
transition to eiDpibyment, ESP staff offered financial assistance for 
various support services, such as child care and transportation. Title XX 
funds covered over 90 percent of these expenditures, while ESP funds paid 
for the remainder. fie estimated value of these expenditures was deter- 
mined primarily from DSS payment records for a represt ntative subsample of 
applicants and recipients. 

For members of the research sample who actually received assistance 
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VIRGINIA 



ESTIMATED NET COSTS OF ESP PER EXPERIMENTAL^ 
BY WELFARE STATUS 



Cost VariebLe 



I 



ESP Opereting Coste^ 

As se scih e hf 

Reassessro ent 

Individ Qfit 
Job S ee rch 

Group Job Sea rch 

Work Experience 

Educetlon and Treining 

Total 
Suppo r t S o rv 1 ce s 
Child Care 
Traneportati on 
Other 
Total 

Community Education 
end Training Services 

Total Costs 



Fu I I Samp I e Applicants 



Red pi ante 



$113 

102 

35 
4S 
68 
21 

3BS 

11 
12 
1 

24 

IB 

430 



$8B 
•'3 

24 

3 2 
27 
7 

251 

1 D 
8 
2 

20 

-1 8 

253 



$129 
121 

43 
61 
95 
30 
47 9 

1 2 
15 

b 

27 

42 
548 



SOURCE: MDRC calculations from VACIS and other program activity data- JTPA 
nformetion systems data and fiscal records; a t t e nda n ce a nd fisca data fr. 
ocal public schools .na ..mmun.ty colle.es; ESP and n:p:r;m " J^^" 
services fiscal records; and MDRC .tuay of ESP staff time allocation 



exp e 
Expe 
are 
pr e- 
cost 
expe 
the 
and 
CO n t 



NOTES The results for op-rating coot, are based on a sample of 2138 
ri.antas and 1044 controls applicants. 1 B97 reeipiants) 

r7 nental-controi differences in the participation rates on ehi^h *k 
.ased were r.greasion-..Justad usi ng rdi n. ry Zlslld^l.^U ^rll 

randon, assignment eh a ra c t a r1 a t i cs of sa« p I a „ e« ba r . . [In plil^^l 
__esti.ations .are based on .... file data for a aubo,„ a o i L " 

ri:^iiT[^;^t:r.'i :^::;trfor"' --r - " ^-^^^^ e.peri..ntai. in 

traininn K . results for support services and community education 

Al 0 t" °" 8Xp.rin,ontals and 

roLs. All costs ere expressed In 1 SB4 dollars. 



_ _ . Because the proportion of staff time speht on reassesfiment 

bi:: irisVIc^i;;:: other activities. the^istin.ti^^TiTc st 

ueiween esp activities are not precise. 
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for child care, the average value of payments during the observation period 
was $553 for experlmentals, and $l|29 for controls. (Controls could receive 
these and other support services without being enrolled in ESP.) However, 
since only a small proportion of the sample received this child care assis- 
tance, the average cost per experimental was only $11 above the $10 mean 
for controls. (See Table 5.^.) For all support services, the difference 
in cost between experlmentals and controls was $2^, adding 6 percent to the 
net operating cost of the program. 

e. Education and Training Costs 

As noted in prior chapters, ESP staff referred experlmentals to educa- 
tion and training programs in the community. Moreover, both experlmentals 
and controls could enroll in such programs on their own initiative. The 
types of programs used included those sponsored by JTPA; community college 
courses; public school classes on GED preparation, adult basic education, 
and English as a second language; and programs offered by proprietary 
vocational and technical schools. 

To estimate the extent to which members of the research sample entered 
these services, a special study was conducted with a representative sub- 
sample of experimentals and controls. As shown in Chapter i| (see Table 
i*.1), this study found that a roughly similar proportion of experimentals 
and controls (16 percent and 13 percent respectively) entered education and 
training programs during the observation period. ^7 However, the experi- 
mehtai-cbntrol difference may be somewhat overstated because VACIS and the 
Fairfax activity records p.^ovided assignment data for experimentals only. 

Since the actual unit costs of the specific education and training 
services used by the research sample could not be obtained, rough 
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approximations (in I98ii dollars) were computed from statewide data. The 
estimated unit costs were $1 ,i|62 per participant in programs sponsored by 
JTPA; per student in GED and other public school courses; and $858 per 

student In community college programs. Determination of the costs of 
education and training services that were only recorded in VACIS and the 
Fairfax activity records was more difficult since the particular kinds of 
services involved were not specified. However, interviews with E^P staff 
suggest that these were mostly public school and community college courses. 
Therefore, a unit cost for education and training services identified in 
VACIS and Fairfax records was computed by taking the average of the unit 
costs for the public schools and community colleges, yielding an estimate 
of $i*86 per person assigned. 

The next step in estimating the average net cost of education and 
training per experimental was to multiply these unit costs by the differ- 
ence in the percentages of experimentals and controls entering the relevant 
activity. These estimates for different types of programs were then summed 
to produce an overall estimate of $18 per experimental, as shown in Table 
5i^i (This low value reflects the fact that these services were not widely 
used by he research sample, and that the experimental-control difference 
In use was small.) For applicants, the estimate of a net cost of $-18 per 
experimental means that the average cost per applicant control was higher 
than the average cost per applicant experimental, which is due to the 
greater use of some education and training servicvss by applicant controls. 
For recipients, in contrast, the estimated cost of these activities was 
higher for experimentals than controls, by an average of $42 per experiment- 
al. Given uncertainty about the accuracy of the information on which these 
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estimates are based, these estimates are not precise. 

When these costs for education and training programs net operated by 
ESP are added to ESP operating costs and expenditures for support services, 
thr resulting estimate of the cost of serving experlmentals — above the 
costs of serving controls is $U3Q per experimental ($253 for applicants 
and $5i<8 for recipients). 

While It Is necessary to estimate costs on a per experimental basis 
to permit comparison with program effects measured in this way — it Is 
important to remember that not all experiment^ls received substantial 
amounts of program services. Indeed, if only the enroilees active In the 
program at some time are considered^ the average cost per person would be 
higher. ESP operating costs were re-estimated for the experlmentals who 
were ever assigned to the program »s activities. (The costs of support 
services and community education and training services are not included.) 
For the full sample, the average gross cost per person increased to 
approximately $603. For applicants it rose to $536, while for recipients 
it increased to $629. 

Vi Distribution of Results 

this section considers the benefits and costs of the program from 
several perspectives, first that of the welfare sample the AFDC appli-. 
cants and recipients included in this study. For this group, the benefits 
generated by ESP include increased earnings and fringe benefits from employ- 
ment as well as increased support services (e.g. ^ child care and transporta- 
tion). However, these gains are offset by an increase in taxes owed on 
earned income, and a reduction in transfer payments. Table 5.5 presents 
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TABLE 5.5 
VIRGINIA 

FROM THE PERSPECTIVE OF THE WELFARE SAMPLE 

ESflMATED BENEFITS AND LdSSES PER EXPERIMENTAL 
AFTER FIVE YEARS, BY WELFARE STATUS 



Component of Anatyel e 


FuL L 


Saro pL a 


Appt 1 ca nt e 


Red pi e nt 8 


Bonaf 1 ts 
















Incraaaed Earnlnge 


9998 


to $1340 


11363 


to $1840 


I8Q6 


to 


$1075 


incraeaed Fringe Benefits 


1 BD 


to 241 


2 46 


to 331 


1 45 


to 


1 94 


Incraaaed Support Services 




24 




2 0 




27 




Lo aaa 8 
















Incraaeed Tax Peymente 


-1 49 


to -201 


-1 80 


to -2 44 


-133 


to 


-17 8 


Reduced AFDC Payments 


"290 


to -359 


-338 


to -422 


-278 


to 


-3 42 


Reduction In bthar TransferB 


-1 00 


to -140 


"1 S3 


to -215 


-82 


to 


-113 


Net Gal n or Loss* 


664 


to 805 


958 


to 1310 


485 


to 


663 



SOURCES MDRe catcntatlons froEj VACIS and other progrero ectlvlty date; JTPA 
InforBBtlon ayatams date and f 1 acs t reco r da ; attehdenci and flacel date from 
local publl cachODta and coramoni ty cot tegea ; MDRC irbrksl te eurvey; Unemployment 
Inauranco ee rni nga a no pay mant a recofdef AFDC peyihentB recbrdaj ESP and 
Departiaent So c1 at Barvlcea fiscal recorde; MDRC etudy of ESP staff time 
allocation; and pubtlahad end unpubllahed de tia oh tax rates, employee fringe 
benefits and a dm 1 n1 s t re 1 1 v e costs for AFOC, Mediceld, Food Stempe, end 
Unemployment Ineuranca^ 



NOTES: Coats were estimated for the aeme five yaer period as benefits 
although most costs were Incurred In the first yeiar efter rendom esslgnmert. 

ReeultB ere oxpraased In 1984 doLLers. The full sample Includes 
2138 experlmentot 8 end 1044 controls (1 285 eppUcehtSr 1 897 recipients). 

Fop benefits, aati mates reflect alter native assumptions about the 
change In impectsofter data collection ended. The first numbir of eech renge 
aasunea that progrem affects decline by 22 percent per yeer during the extre- 
poletlon period; the second number assumes rib decey or Increase. 

- - - 

the net gein or loss Is the sum of ell benefits end losses. 
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the benefits and losses estimated for the five-year time period and on the 
bottom line shows the bverala results of adding them together^ (the esti- 
mates are calculated on a base that included nbnparticiparits as well as 
participants,) Experimentals enjoyed a net gain as their increased 
earnings clearly outweighed their increased taxes and reduced transfer 
payments. For the full sampie, estimates of this gain range in value from 
$66i* to $905 per experimental over five years^ depending on which 
assumption about the rate of decay in program effects is used. (The lower 
value reflects ah annual decay rate of 22 percent during the extrapolation 
period; the higher value assumes no decrease or increase.) The net gain is 
larger for applicants than recipients ($958 to $1310 and $485 to $663 
respectively) , although applicants lose more in taxes and transfer pay- 
ments. These greater losses are outweighed by applicants' increased 
earnings and fringe benefits. 

The government budgetary perspective is of critical concern to policy- 
makers interested in budget savings. According to this perspective, 
increased tax paymeuts, reduced transfer payments, and reductions in the 
costs of administering transfer payments constitute benefits. 3^ In 
contrast, government dollars are expended to operate ESP, as well as for 
support services and education and training programs in the community. 
Table 5.6 presents these gains and losses and shows that, for the full 
sample^ ESP resulted in savings for the government budget ranging from $159 
to $336 per experimental. Most of these savings are due to the effects of 
the program on applicants — a net gain from the budgetary perspective of 
$506 to $7^6 per applicant. For recipients, estimates ranged from a net 
loss of $29 per experimentai to a net gain of $119, dependirjg on 
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TABLE 5.6 
VIRGINIA 

FROM THE GOVERNMENT BODGET AND TAXPAYER PERSPECTIVES 



ESTIMATED BENEFITS AND tOSSES PER EXPERIMENTAL 
AFTER FIVE YEARS, BY WELFARE STATUS 



CompofiahS of AriaLyal s 



I 



Fu t I Ssmp t e 



Appt 1 ce nt 8 



Reel pi ehts 



Benef Its 

Increased Tax Payioents 
Reduced AFDC Payments 
Reduction In Other Transfers 



Re do ce d T r s h sf e r Program 
Adm 1 n1 s t r a t1 on 



Lo 8sa s 

ESP Dparatlng Costs 

Support Service Costs 

Cbnmuhl ty_ Educe t1 on and 
Trslhlng Costs 



1149 to $201 

290 to 359 
100 to 1 40 

50 to 6 6 

-3BB 
-2 4 

-1 B 



SiBd to $244 

338 to 422 
1 53 to 21 5 

BB t o 1 1 B 

-251 
-20 

1 8 



$133 to $178 
278 to 342 
82 to 113 

2 6 to 3 4 

-479 
-27 

-42 



Net Gain or Loss from the 
Budget Perspective^ 



1 59 to 336 



506 to 746 



-2 9 to 119 



In-Prbgrem Output {Benefit) 



1 04 



41 



1 45 



Nat Gain or Loss from the 
Taxpayer P e rspactlve^ 



263 to 440 



5 47 to 787 



116 to 264 



SOURCES See Table 5.5. 



were Bstlneted for the iame five year period as benefits 

J "^"''''■^ J " the first year after random esslgnmenti 
Reaults are axp r eased In 1 9B4 dottars^ The fuLi aample Iricludis 213B experl- 
roentals end 1044 cbntrola {1285 appticants, 1897 recipients). For benefits 
other then In-program output* estlnatae raftact atternatlve assumptlbhs about 
the chahjsa In Impacts after data collection ended. The first nQober of each 
range asaumes that program affecta decline by 22 percent per year doring the 
ixtrapolatloh period; the second number asaumaa no decay or Incraasa. 

a - - 

The not gain or loaa from the budget perspective is the aum of 
ell baneflta and tossiiSr axcludihg the velue of 1h-prbgran output. 



The net geln or loaa from th'a taxpayer perspective la obtained by 
adding the value of In-program output to the net gain or loss from the budget 
perspective. 
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assunptions about decay. Given this range of estlinates and the various 
decisions made in calculating them, these results are probably best 
interpreted to mean that, for recipients, the government budget essentially 
••breaks even" after five years; 

the effect of ESP on the goverriment budget is quite close to its 
effect on taxpayersi31 two perspectives are not identical since 

taxpayers benefit not only from the net savings to the government budget, 
but also from the value of output produced by ESP participants in work 
experience assignments^ taxpayers benefit from the services performed by 
particip>ahts assigned to work experience without having to pay for them. 
Adding the value of services performed in work experience to the estimated 
budget savings yields the overall gain to taxpayers: $263 to $i*40 per 
experimental^ with a positive value for recipients as well as for appli- 
cants, (See Table 5,5 •) 

A final perspective considered here is that of society at large, which 
includes both the welfare sample participating in this study and taxpayers. 
From the perspective of society as a whole, program effects that are a gain 
to one of these groups but an equivalent loss to the other group yield no 
net benefits; they are simply transfers between groups^ For example, the 
reduction in AFDC payments is a loss to the welfare sample that is offset 
by equal savings for taxpayers. In contrast, the reduction in the admin- 
istrative costs of the AFDC program is a net benefit to society as a whole 
since taxpayers save money and the welfare sample is not directly affected. 
Similarly, increased earnings of the welfare sample are not fully offset by 
losses to taxpayers. 32 (see Ajppehdix Table G.3 for further detail on the 
distribution of specific categories of benefits and costs.) For the full 
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sample^ the total benefits to society were estimated at $1|333 to $1*751 i 
while estimated costs totaled $if06, yielding a net societal gain of $927 to 
$1,3^5 per experimental after five years. The estimated net gain for 
applicants ($1,505 to $2,097) exceeded the net gain for recipients ($601 to 
$927)- 

The overall gain or loss generated by ESP from isach of the four 
perspectives discussed above can be compared in Table 5.7 for applicants, 
recipients and the full research sample. From almost all of these 
perpectives, the economic benefits of the program exceeded the losses it 
generated. The one exception is for recipients from the government budget 
perspective. For that group* whether or not ESP resulted in net budgetary 
savings depends on assumptions about the decay rate. 

As noted throughout this chapter* the results of the benefit-cost 
evaluation of ESP were calculated assuming that program effects continued 
to occur after the observation period — either unchanged or at a decay 
rate of 22 percent. Because the extrapolated effects constituted a major 
share of the total effects estimated for the five-year time period (see 
Section III.E), it is useful to consider how the overall benefit-cost 
results would change if program effects are assumed to be zero after the 
end of data collection. Such an assumption would yield lower-bound 
estimates of the program's results. 

To test the sensitivity of the benefit-cost results to this assump- 
tion, ail benefits and losses were re-estimated without including the 
extrapolated effects. The overall results, which are presented in Apj>endix 
Table G.i|, reveal that the welfare sample's gains from the program still 
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TABLE 5,7 



VIRGINIA 

ESTIMATED NET GAIN OR tOSS PER EXPERIMENTAL FIVE YEARS, 





BY WELFARE 


STATUS AND 


PERSPECTIVE 






I 

Perspecti ve 




Wet fs re 








i^ei fe re Ste tue 


Seropl s 


Budge t 


Taxpey e r 


Soci ety 


Fu I I Sam pie 


664 to 905 


1 59 to 336 


263 to 440 


927 to 1345 


Appl icents 


S58 to 1310 


506 to 746 


547 to 787 


1 505 to 2097 


Reel pi ente 


4 85 to 66 3 


-29 to 119 


116 to 264 


601 to 927 



SOURCE: MDRC cetcutetions f r om V ACIS e nd o th e r progrem activity date; dTPA 
Information systems de te end fiscet records; attendance and fiscal date from lociaL 
)ublic schools and community colleges^ M5RC v orksi te 8 q r v ay ; Unem p I oy me n t Insurance 
sernings and payments records; AFDC pay me nt s r aco r ds ; ESP and Department of Soci eL 
Services fisceL records; MDRC study of ESP s t af f t i me a t t o ce ti o n , end pobtished end 
jfipublished de te on tex rates^ employee fringe benefits^ and administrative costs fc 
iFDCff Medi cei df Food Stemps, end Unemployment Insurance. 



NOTES: Within eech perspective* positive numbers indiceto gains to thet group 
ind negetive numbers indicete Losses. 



Results ere expressed in 1984 dolLersi The full semple includes 2138 
xperinentels and 1044 controls M285 eppLicentSf 1897 recipients). 



Estimetes reflect elternative essumptions about the change in progrem 
ffects efter deta collection ended. The first number of eech reng^ assumes that 
rogrem effects decline by 22 percent per yeer during the ex t r epo I e t i on period; the 
econd number essumes no decey or increese. 
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exceed its losses. In contrast^ for the governoeht budget, the losses 
outweigh the benefits. From the taxpayer and societal perspectives, a net 
gain was estimated for applicants while a net loss was estimated for 
recipients. Overall, these findings support the conclusion that ESP is 
beneficial for applicants and recipients. However, the program cannot fae 
viewed as cost-effective for the government budget or taxpayers unless 
future prograifl effects are considered and are assumed not to decay 
substantially. 

This benefit-cost analysis has not taken into account several 
important factors worth underscoring. The higher employment rate of 
experimentals could result in some displacement of other workers^ yet this 
evaluation assumed no displacement occurred. The analysis also did not 
assess such an intangible benefit as potential enhancement of self-esteem 
as a result of working. In addition, the evaluation did not consider the 
implications of welfare mothers spending more time working and less time 
with their children. Mor did it take account of the degree to which 
society values working over receiving welfare. The factors not weighed in 
this analysis should be considered in interpreting its results. 
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TABLE A^l 



VIRGINIA 

DISTRIBUTIdN OF APPtiCANTS AND RECipiENTS 
RANOOMtY ASSIGNED. BY REASON FOR EXCLUSION FROM RANOOM ASSIGNMENT 
(AUGUST 1983-SEPTEHBER 1984 SAMPLE] 



Reoaon 


Appl 1 ca nts 
and Reclplehte 




Currently Active in ESP^ 


4.3 




CurrinntLy SchoduLed to Begin 






aeiectea tSr Cob po ne n t e 


6.9 




Prior Pa r a ni TiK t.-i nn in fialM^^oiHl 






ESP Cbnponents 


3.9 




in Fott-Tlroe No ESP Educetlon or 






Trai ni ng 


14.0 




Mendatory Registrant With Child Less Than 






Six Veers Otd 


59.5 




Vol uii t ee r 


8.7 




e 

Me L e 


2.7 




Total Number Not Randomly Aeslgnad 


2995 



SOURCE: Celculetlons from MDRC Client Infbrmetlon Sheets. 

NOTES: Some Indlvlduels ware excloded from rendom essignment 
for more than one reason* 

e~ _ __ _ ILL 

^«c^P^o"t6 were screened for carreot pe r ti ci pa t1 on In 
ESP components: group job search, woFk experl ence, trel ni hg end 
oducetlon. Individuals who were currently participating in individual 
job search sere considered aliglble for random assignment, 
b 

Racipitnts who were schadul ed within thirty deys to 

begin ESP work 6xperi«nc«, education or training ware not aUglbla for 
randoi assignment. 

c __ _ _ 

Reciplanta who participated In ESP iork experience, 
aducatlon or training ware not aligibta for random assigniant. 

d 

SI ngle parent* who have childrian under six years of age 
on their case are ilN-mandatory whan they era out of the home for more 
than brief or infrequent periods. 



e 



inadvertantly randomly assigned to th _ 
raseerch groups. For thia analyeia, they were considered to be pert 
of tha non- rasea r ch group. 
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VIRGINIA 



SEtE:::EO CHARACTERISTICS OF THE ESP ENROLLEES 
AT THE TIME OF RANDOM ASSIGNMENT, B Y R ESE ARCH SAMPLE GROUP 
UUGUST 1983 - SEPTEMBER 1984 SAMPLE] 



Cho ra c t • r1 St 1 c 



AFOC Statub [X] 
Appl 1 cih t 
Rtcl pi iht 

Aviraga Age [Yaars] 

Ethhl c1 ty [XJ 

Vh1fa» Noh-Ki spahi c 
Bl.ackt Non-Hl8pahic 
Hispanic 
Othar 

Oagraa Recaivad [%] 
Nona 

Ganarat Equlvatancy Diptooia 
High School biptoaa 

Prior AFOC Oapandancy [%] 
Nav er bh AFDC 
Two Yaarc or Less 
Mora Than T« o Yaars 

Hal. d Job at Any Time During Four 
Quartars Prior to Rendoa 
Aisl ghoaht {%] 

Avaraga Earnings Ourlhg Four Quartars 
Prior to Randoa Asslgnaant (Sj" 



Totat Sanpla 



I 



"ch Sampla 



40.4 
59.6 

33.6 



32. B 
64.0 
1.2 

2.0 



56, 
8. 
35. 



12.1 
28.1 
59. B 



36.7 



1167.56 



3164 



Non-Raaaar ch 
Saa pi a 



23.5*»* 
76.S»»» 

27. B*** 



25.0»»» 
72.2»»» 

0.6 
2.0 



45 .2»»» 

6.1 
46.7**» 



1 0 . 2 • • 
34.B»»» 

55. O*** 



36.0 



741 .S5**» 



2786 



SOURCE: Catcgtatlona fron MORC CUaht Inforaatlon Shaats and UI aarnlngs 
racords f roo tha CoavonMeatth of Vl.glnli. 



NOTES: Di strl butlona aay not add axactly to 100.0 parcant becausa of 
roonding; 



Catcotataa froa Uhaapl byidaht Ihauranca aarnlnga racords froa the 
Coaaonwaalth of Virginia. 

- - - - 

^or aalactad cha racta r1 at 1 ca» aaapla sizas aay vary op to 36 
sampta points dua to ilsslhg data. 



01 f f a ranee 6 &e t« Co h •aapta groups ara statistically significant 
using a twp-tailad t-tast or chl-aqoara tast at the fotloslng tevals: • ^ 10 
parcant; e. 5 percent; » 1 parcant. 
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TABLE A. 3 



VIRGINIA 

Sa.ECTEO CHARACTERISTICS OF THE RESEARCH SAMPtE 
AT THE Tim OF RANDOM ASSIGN^ENTr BY RESEARCH GROUP 
(AUGUST 1983 - SEPTEMBER 1984 SAHFtE] 



Chiiracterletic 


Controls 


Job Seerch- 
Work Experience 


All ESP Services 


ToteL 


Welfare Stetus (X) 
Apptlcsht 
Reclpisht 


41.1 
58.9 


39.0 
61.0 


41.1 
58. S 


40.4 
59.6 


Avsfage Age (YeerRj 


33.3 


33.9 


33.7 


33.6 


ethnicity l%] 

Whiter Noh-Hi8pahic 
Btackt Non-Hlspenic 
Hlepanlc 
Other 


32.0 
64.3 
1.3 
2.4 


32.9 
64.0 
1.2 
1.9 


33.6 
63.7 
1.0 


32.8 
64.0 
1.2 
2.0 


Degree Received (%] 
None 

Seneret &|Qlvatency OlpLome 
High SchDOt Diptane 


55.1 
7.7 
37.2 


57.4 
7.9 
34.6 


56.5 
8.8 
34.7 


56.3 
8.1 
35.5 


rrior ArDU liependency {%} 
Never oh AFOC 
Tno Years or Less 
Mora Than Two Yeers 


12.7 
27. B 
59.5 


12.4 
28.3 
59.4 


11.3 
28.1 
60.6 


12.1 
28.1 
59.8 


Held Job at Any Tine During Four 

Qtfarters Prior to Rendbm 

— e 

Asslghmehte (Xj 


38.1 


33.8 


38.3 


36. 7'^ 


Averege Earnings Olirlhg Four Qiierters 

- - . __ . - - - fl 

Prior to Random Asslghmeht ($] 


1237.86 


992.71 


1271.67 


1167.56** 


Totet Senipte 


1046 


1061 


1077 


3184 



CeUutetldns from MDRC Client Inforraetibn Sheets end Ul eernings records from the Coninoweatth 

of VI rgl n1 a. 



NOTES: Dlstrlbutlbhs nay hot edd exactly to 100.0 percent because of rounding. 

e- 

CalcLilated froin Uhenployaent Insurance earnings recorde fron the CcoifnonHreet th of Virginia. 

b _ _ 

For aalectad charecterlBtl C8» samplis alzee ney vary up to 18 sample points due to missing data. 

O^ffcrencea acrbSB raeearch groupe are istati etically significant using e taio-teiled t-test or 
chi-squere test at the following lavele: • = 10 percent; = 5 percent; = 1 percent. 
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TABLi A.4 



VliRGZNIA 



SELECtiEb CHARACTERISTICS OF APPLICANTS 

AT THE TIME OF RANDOH ASSIGNMENT! BY PERIOD OF RANDOM ASSIGNMENT 
(AUGUST 1983 - SEPTEMBER 1984 SAMPLE) 



Cha racte r 1 6t1 c 


1 

August 1883- 
March 1984 


1 ' 

Apri l-Septeaber 1984 


tocet Agency [%] 
Fai pf ax 
Newport News 
Hampton 
Cheea pe a ke 
Rural Age ncl e 8 


27.7 
32.6 
If). 7 
3.5 
20.5 


27.1 
20.8*** 
13.1 
16.4*** 

22.5 


Avarage Age (Years] 


32.9 


33.2 


Ethnicity [%] 

Whitet Nbh-Hlspehic 
Blacky Noh-Hi spa ni c 
HI apani c 
0th e r 


41 .2 
55,7 

i .5 

1 .6 


42.3 
54.0 
1 .5 
2.2 


Degrie Received l%] 
None 

General Equivalency Oiploma 

ni)|ti Oi«nuwL wipL Oili B 


50.7 
8.3 
41 ^0 


47.0 
8.5 
44.5 


Prior AFDC Dependency (%] 
Nev er on AFDC 
Two Years or tees 
More than Two Years 


23.3 
32.7 
44^0 


30.1 *♦* 

29.9 

39.9 


Held Job at Any Tine 
Daring Feur Quarters Prior 
to Random ARSlgrnent 1%]^ 


56 .2 


58.6 


Average Earnings During 
Four Quarter Prior to 
RandQB Aaslghaieht [$] 


1997.31 


2695.40*** 


Sample Size 


745 


542 



( CO n t1 n uad ) 




Tebl e Ai 4 [conti nged ] 



SOURCE: CBLcuLetibne frba tha MDRC etlont InforDatloh Sheets and UI 
08 rn Inge records from the Cowmbnweelth of Virglnlai 

NOTES: Distributions ney not edd exactly to lOOiO percent becBuee of 
roon d 1 ng. 

CelcuLatBd from UhBoployaent InaijrBncB BarnlnflS racords from th 
Cbnabhwaalth of Virginia. 

b- 

BBlected ch a r e ct a r 1 b t 1 cs * BampLe sizes mey vary up to B 
GBRipte points due to mlsiBlhs dete^ 

DiffirBncBB batwaen parlods of random assianment are 
ete ti Btl cat ty si gnl f 1 coht UElng a two-tBlled t-test or chl-oquara test at the 
fotlowlng levels: ♦ == 10 percent; = S percent; ♦ * = ^ percent. 
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TABtE A. 5 



VIRGINIA 



SELECTED CHARACTERISTICS OF RECiPlENTS 
THE TIME OF RANDOH ASSIGNMENT, BY PERIOD OF RANDOM ASSIGNMENT 
(AUGUST 19B3 - SEPTEMBER 1984 SAMPLE) 



Cha racterletic 


1 

1 August 
1 March 


1 8B3- 
1 884 


1 

Apr i L-Septembe r 1984 


Locet Agency (%] 
Fei rf e X 
Newport News 
Hen pt on 
Chase pe e ke 
fiu ra L Age ncl e s 


j 14.4 

J 14.6 

1 SO i9 
I 21 ,3 




29.1*** 
20. 6*** 
16.2 
12.1*** 
20.0 


Averege Age [Yeere] 


I 34.2 




33.1** 


Ethnicity (X] 

Whiter Nbh-Kiepanic 
Btack» Ndn-Hl8pen1c 
HI Gpeni c 
Other 


SBiO 
70.0 
1 .9 
0.1 




21 .6** 
71 .0 

1 .2 

6.2 


Degree Received [%] 
N 0 n 0 

Sanerat EqulveLehcy OlpLome , 
High School OlpLbine 


61.3 
6.3 
30.5 




61 .1 
6.6 
32.0 


Prior AFDC Depenaehcy l%] 
Never on AFOC 

TvD Yeere or Lees 1 
More then Tib Yeere 1 


i .5 
26.9 
71.6 




7.4*** 
20.4** 
72.2 


HatdJDbet Any Tine 1 
During Four Quarters Prior J 
to RandDBi Assignment l%] j 


22.6 




23.2 


Average EarnlngB During 1 
Four Daerter Prior to 1 
Randon Aetignmeht ($] | 


427 .29 




310.63 


Sample Size^ j 


1557 1 


340 



( conti nued ] 
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TABLE A. 5 [continued] 



SOURCE: CeLcuLatibriiB frbn HDRC CLIiant Information Sheets end Ui eernlnge 
records froK the Cbm«QnweeL th of Virgihie. 



Catcutrted frbro UnempLoyment Ineuriarice eHrnlngs records from the 
CooinonweaL th of Virginia^ 

b 

Po** select -1 charactiSrl sti csr eerapLe sizes oay vary up to 6 
saoipte polnte due to alsb ig data* 

Differences betweisn perlbds of rendom eBSignment ere 
sta 1 1 sti ca L Ly signlflceht using a two-tailed t-iest or chl-equare test at the 
fottowlng LeveLe: * = 10 percent; •* = 5 percent; *** = 1 percent. 



NOTES: 
roundi ng. 



D1 G tri buti oh 6 may 



not add exactly to 100.0 percent because of 



a 




im A.6 



VIRGINIA 



WINTHLY UNEHflOY«NT RATES, BY LOCAL /CBlCY 
lAUUST m-M 1985] 







Urban Agencies 




1 

Rural AganclBB 


1 






I-- 


1 


i ' 

1 

ChflBB- 




1 


1 




IJii. . Ill a. _ 


1 ' : 
PlttByt- 


1 




Hontn 


Fal IT ax 


Nave 


Hflnihthh 

11(111 UwUII 




Hanri/ 

nBnry 


uroy vun 


OBIQX 


uerrou 


vlUe 


venla 


--- 

Cempbett 


VlrgiiHe 


AQgoot i9S3 


2.6 


4.3 


4.6 ■ 








DiD 


OfU 


Oil 


D 1 
D,7 


4,2 




Septsiiber 


2.5 


4 4 


48 


4,9 




R ^ 

Diu 






C 0 

o«2 


5.6 


4.4 


4 fl 


October 


V 


5 0 


5'B 


4,8 






^ h 


0.1 


>i 0 
4iD 


7.2 


4.9 




Nov fiber 


2.7 


5:1 


'" 5.6 


4.7 


U|U 


7 7 


D 


U|0 




Old 


7.2 


5.2 
Wife 


bflcenibar 


2.6 


4.8 


5,1 


4,7 


6.8 


7,9 


49 


5 7 




7 P 


c 7 
0./ 


5.3 


Januery 1984 


2.8 


5,3 


5.6 


5,1 


y |U 


9 9 


^1 7 


R 7 


Old 


Q 7 


7.0 


8.2 


February 


2.5 


4,9 


5.5 


4.7 


8.6 


9.1 




7 R 


R 

OiO 


llll 


7.1 


5.7 


Horch 


2.6 


5.1 


5.4 


4.4 


5.2 


8.9 






5.9 


7.0 


5.4 


5.4 


Aprli 


2.4 


4,7 


5.0 


4.6 


5.4 




4.5 




5,D 


6.5 


5.4 


5.0 


fty 


2.6 


4.6 


4.4 


4.0 


7;2 


B;2 


4:1 


W|U 






4.4 


4.8 




D-Q 


4,6 


4.8 


4.B 


4:7 


B'5 

U lU 


4 5 






R 0 


4.C 


4.9 


July 


2.8 


4,8 


5,0 


5.2 


6.6 


9i1 


6:9 

W|0 


8:3 




Q ^ 

□ lO 




5.2 


August 


2.5 


4.4 


4,6 


4.3 


5,6 


8.0 

U|U 


4 P 










4.4 


jeptenbGr 


2.5 


4.4 


4,7 


4,2 


5.4 






BP 




R 1 




4.4 


October 


2.7 


S,D 


4,9 


4.5 


7.1 


8.7 








8,8 


6.6 


4.B 


November 


2,5 


4,6 


4.5 


4.0 


8.9 


7.7 


3,6 


B.I 




9.6 


7.6 


4.8 


December 


2.7 


4.8 


4.6 


3.B 


7.8 


8,3 


4.2 


9,8 


6,9 


9,4 


6.5 


5,0 


Jenoory 1SB5 


3.2 


5,3 


5,5 


4.4 


0.0 


10.8 


6,5 


11.4 


7,2 


9,9 


7.2 


5.9 


February 


2.6 


5.2 


5,5 


4.8 


10.0 


13,0 


8,5 


14.4 


8,2 


11,5 


9.4 


R ii 

9.0 


Harch 


2,5 


4.5 


5,0 


4,2 


8.7 


11,9 


5.8 


14.0 


7,0 


9,8 


7.3 


5.3 


April 


2.5 


3.8 


4,3 


4,5 


6,5 


12,0 


5'1 




6,0 


7.9 


5.7 


4.B 


ftoy 


3,1 


3.9 


4,7 


4,9 


6.4 




6.1 




6.2 


8,1 


6.1 


5.1 


June 


3,5 


5.0 


5,1 


5,3 


5.8 


12.7 


7.6 


9,6 


5,6 


__J.7 


6.7 


5.4 


Average 


2.7 


4.7 


5.0 


4,6 


6.6 


9,3 


5.3 


7,8 


6.0 


8.1 


6.0 


5.1 



SOiRGEj 
NOTES: 



Ceicuifltjons from the Virginia Employiiient ComniiEsioni tabor Fores EetliBtBB. 
Uneiplbyinent rates are not EeaEonaUy actjueted. 
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APPENDIX B 
QUALITY CQNTRQL 



!• ^Ileirt^^Hformatlon Sheets 

As part of the effort to maintain high quality in the data recorded in 
the Client Information Sheets (CIS), HDRC trained DSS staff in administer- 
ing the questionnaire and during site visits monitored interviews in which 
the questionnaire was administered, MDRC checked the quality of the data 
by contrasting self-reported information oh prior employment and AFDC 
receipt in the CIS with UI records and VACIS; comparing the distribution of 
answers to those given In other programs included in MDRC's Demonstration 
of State Work/Welfare Initiatives; searching for and excluding impossible 
answers (such as having more than 20 children under age 18); and identify- 
ing whether any DSS office provided a large share of incorrect answers or 
missing data. As indicated in Chapter 2, this quality control check showed 
the CIS data to be at least as accurate in Virginia as in any of the other 
states included in MDRC^s Demonstration, 

For 30 people (about one percent of the sample), data were missing for 
one or more of the CIS variables used in the process, impact or bene- 
fit-cost analyses. Data were missing for only one or two of the variables 
for about half of this group, and for all of the variables for the 
remainder. 

All available demographic data were used to examine patterns of 
participation. For the analysis of impacts and benefits and costs ^ missing 
CIS data were imputed according to the following procedure. Subgroups of 



the research sample were formed using data that were avallablie for every 
member of the research sample: research group, AFDC status (applicant or 
recipient), DSS office, and prior work history (ever employed or never 
employed during the year prior to random assignment). These variables 
could be combined in 132 ways and the researchers fit all research members 
for whom data were missing into these subgroups. For each subgroup, the 
researchers ascertained the answer most frequently given to each of the 
questions on the CIS for which data were missing and substituted these 
modal values for the missing data. This procedure was used for all members 
of the research sample who lacked data — except for the two cases missing 
Social Security numbers, who therefore also lacked earnings data. (Social 
Security numbers were used to reference earnings data from the computerized 
files in the state Unemplojrment Insurance system.) Therefore, the 
regression equations for the impact and benefit-cost analyses both use a 
sample of 3il82. 

In order to test the accuracy of payments data in VACIS and on the 
record forms sent by staff members in Fairfax County, MDRC conducted a 
quality control check of payment records for a subsample of 1 57 members of 
the research sample from the four urban agencies who at some point received 
welfare benefits while working. mbRC had gathered these data for a 
previous study (Goldman et al., 1985a). Although this subsample was not 
representative of the entire research sample, it was appropriate to use in 
testing the accuracy of the payments records in VACIS and in the Fairfax 
County reporting system. 
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For the members of this subsample, the researchers compared the valiae 
of payments obtained from VACIS or the Fairfax County forms to the 
corresponding value of AFDC payments contained In the case flies. The 
comDined total of case months examined was 1,41 3, MDHC found that the data 
in VACIS failed to match payments data In the case files for slightly over 
ID percent of the cases. The mean absolute difference between 
corresponding payments was about When positive and negative values 

were used and the values of payments from the case file subtracted from the 
computerized analysis file, the resulting mean difference was -$1.88, 
Indicating that the payments data in th^ analysis file slightly 
underestimated the actual value of registrants* welfare grants. 
Differences among research groups were not significant. As indicated in 
Chapter 2, these results were consistent with those from similar quality 
control checks pe: formed for other state information systems used in MDRC 
studies of state work/welfare Initiatives. Therefore^ the researchers 
decided to use the payments data In the analysis file without further 
modification. An examination of the distribution of the values of monthly 
payments In the analysis file did not turn up any payments so large as to 
be unlikely. 

III. UI Earn ings an^ Be n e fits 

Quality control checks of data in MDfiC's analysis file drawn from 
Virginia's UI System revealed a close fit. MDRC compared the value of each 
quarter of earnings and each month of benefits in the two data sources for 
a randomly selected subsample of 50 registrants who were recorded on MDRC's 
analysis file as having earned money during the third quarter of 1984, as 
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well as for a second group of 25 people who were listed as never having 
worked during that quarter or any previous quarter up to and Inclucilrig the 
quarter of random assignment. The data quality test found a discrepancy in 
the earnings data for oxily one member of the subsampla and discrepancies in 
the UI benefits data for two others, C3ivan this low error rate, the data 
in the analysis file were used without modification. 

As an additional check on the quality of the data, MDRC examined the 
distribution of earnings and UI benefits for ail members in the research 
sample to identify clearly impossible data for quarterly earnings or 
monthly benefit?. Since no impossible value was found, the earnings and 
benefits data were used without modification. 

In contrast to the information on AFDC payments and program 
participation, the UI earnings and benefit data originated from a single 
statewide centralized data base and therefore were not subject to 
systematic interagency variations related to different methods of 
recordkeeping. 
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APPENDIX C 




T«tE C;1 



VSIGINIA 



ALL AFDC: NINI-HDNTW rtRTORHANCE ioiCSTOBS FDR 
EXPERIHENTAtSi BY tOOLJGBlCy AND RESEAflOi GROUP 
tAUeUST 19B-8EP[EWEn 19W SAfiPLEl 





Fai rf ox 


1 

Nwpdrt HmB 


1 

Hmpton 


1 

Cheeepeake 


1 " 
Roral 1 


S^gehclee 




Job 

Search- 




Job 

Sfiapctr- 


1 


Job 

Seirch- 


1 


Job 

Seercii- 




Job 

Search- 


i 


ftirforRiahCB Indicator 


Work 
Experi- 
ence 


Att 
ESP 
ServlcsG 


ihce 


All 
Servicee 


Wbrk 
experi- 
ence 


All 

CCD 

Services 


Noric 
ExpeM- 
enca 


All 
ESP 
Services 


Work 

ixpepi- 
enra 


M 

ESP 
Services 


^- 


80.1 


78. I*** 




DO.U 


76.3 


85. 4* 


93.3 


89.4 


SB. 9 


86.9 


Ever Active 


67.1 


5i.Q»*» 


42.0 


44.9 


57.1 


ee.gM 


67.5 


6B.8 


69 3 


UWIIJ 


Participated In Job Search 
Individual Job Search 
Group Job Search 


Dq.4 

5S.B 
2.6 


46i6** 
45.5** 

5.7 


34.B 
10.2 
27.6 


36.1 
10.2 
31.0 


44.8 
36.5 
9.6 


5B.7** 
53.0*»* 
8.1 


60.1 
46.0 
25.8 


66.9 
53.8 
24.4 


65.3 
64.4 

2./ 


56.3* 
S4,1M 

5.2. 


Participated In Rork 






















Exparlerice 


1/.0 


Q 13 AAA 


8^2 


3.7*» 


23.1 


9.8«M 


17.8 


13.1 






Participated In Training or 






















Education 
Trai ni no 
Educati bh 


3.9 
2.2 
1.7 


4.4 
7.0*« 


1 1 .a 

2.7 
9.6 


170 
i/.U 

4.4 

12.6 


8.3 
3.8 
4.5 


14.6 
3.0 

11.6«» 


11.0 
4.3 
6.7 


11i3 
4i4 
6.9 


9.8 
4.9 

4;9 


14.8 
8.3 

U.v 


Deregl stored'' 

c 

Due to Sanctioning 


46.3 
7.8 


64.5*** 

8.0 


15.7 
1.4 


16.0 
0.3® 


22.4 
3.8 


29.3 
2.4 


46.6 
4.9 


41.3 
5.0 


2B.B 
2.2 


34.1 
4.4 


Placeoieht Rate^ 


'U7 


0.0® 


9.9 


8.5 


17.9 


19.5 


3D.1 


33.8 


16.4 


19.2 


Total Number of 






















ExperiirentalE 


224 


230 


283 


294 


156 .164 

^ ,1^, 


163 


160 


225 


229 



"1 — — — 

(cbnti huedl 
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SBJHCEj CatciiletlonB frc* the VlPBlnia Autcniated Client infopiiiatibn SyBtem and froni program ecthlty date cottected from case 
file recorda of a random subsBmple of experlmentaU in Fairfax Countyp 

NOIES: Perfbrinance Indicatore are defined aa ever eeellined to a partlcutar cotnponent or etstua. 



Pirfbriance indicators are calculated as a percentage of the total persons in the indicated reeearch group In ett 
agehclfifl biit Fairfaxi in Fairfax County^ perforfnance indicators are ralculetsd as a percentage of a random stibBempte of 1B3 
experimental 6 ihboa case f ilea ware reviiad. 

B- _ .;_ .._ _ 

Aseeeoiieht la defined for eppUcantB and redeterniined XiN-mandatory recipients as contact nith an Biiptoyinent Services 

(Jbrker after randbiti aeelgrHnent. For those reclpiente liho nere WiN-mendatory before the atort of the researchj asBesgnent Is 

defined as re^ssesaiieht at the point of randoni assignmerit. 

b . 

Dflr'bgi strati on is defined es being der^istered according to ESP program recorda or feiting to receive AFDC payinente 

at any time during the nine inonths folloilng random assignment. 

c^ 

Sanctioning rates m defined as referral for sanctions, 

d __ 

Program placement information is besed on emploj'flent that is reported to program staff. Progrem ptecement feta till 

not be used to measure impacts. 

0 

Chi-square test ineppropriete due to low expected cell frequencies. 

A chi-squsre test m sppUed to differantts between research groups within local egencies. SteticLicst significance 
levels were calculated es: ♦ = 10 percent; = 5 perwnt; and = 1 percent. 



TABLE C.S 
VIRGINIA 

AFDC APPLICANTS AND RECIPIENTS: DISTRIBUTION OF EXPERIMENTALS 
IN PROGRAM ACTIVITIES^ BY WELFARE STATUS 



(AUGUST 1 863 


- SEPTEMBER 1 9B4 


SAMPLE] 




Acti V 1 ty 


1 

App L i ca n t 8 


Recipients 


To teL 


Never Active 


58. 3 


30.6 


41 .7 


Job See rch Only 


30. D 


43 .2 


37.9 


Poet Job Seerch Services 


11 .7 


26.2 


20.4 


Work Experience Only 




3.6 


2.3 


Educe tidh and Training Only 


4i1 


5.3 


4.8 


Job Seerch and Work 
Experi ence 


2.5 


9.1 


6.4 


Job Search and idocstion 
and Training 


4.4 


7.2 


6.1 



Work Experience end 
Educe tion and Treininq 

Job Seerchr Wo r k Expe r i ence , 
and EducB t i on end frai ni h g 



To teL Number of ExperiroentetB 



0.0 



0.4 



857 



0.3 



0.7 



1 281 



0.2 



0.6 



2138 



SOURCE: Cetcutetions from the Virginia Automated CLient Informetion 
System and from progrem ectivity da ta coL lected from case file records of e 
random eubsampLe of a x pe r i me n t e t s In Feirfax County. 

NOTES: Performance indicetore ere defined es ever essigned to e 
perticuLar component. 

^'^P^''^«entat s are comprised of both Job Seerch-Work 
Experience and All ESP Services research greops. 

Tests of stetisticet eignificence between eppLicsnts end 
recpients were not exemined. 
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TABLE e.3 



VIRG INIA 



NINE-MONTH PERFORMANCE INDICATORS FOR ASSESSED EXPE R I MEN TAL S, 

BY WELFARE STATUS 

(AUGUST 19B3 - SEPTEMBER 1384 SAMPLE) 



Performence Indicetor 



Ever Activb 

Participated In Job Search 
IhdIvldueL Job Seerch 
Grbuj) Job Saerch 
Acti V 1 11 es 

Per ti c1 pa ted l n Work 
Ex p ja r 1 e nee 

Partlclpeted In Trelning 
Of Edii ca t1 on 

Trelning 

Educe t1 on 

Der egl stered^ 

Due to Senct 1 onl ng*^ 

Place d^ 

Total Nuinber of 
Aaae Bsed Expe r1 mente L s 




TOtBt 



73.0*** 

63. 9*** 

50.6* 

1 8. 5** 



11 .B*** 



14.6 
5.3 
9.3 

39.9*** 
4.3 

17. 7*** 



1704 



_ ^SOURCE: CelcuLetlons from the Virginia Autometed Ctieni Informetibh Svstem 
and fro. program ectlvlty dete cottact.d from ceee file records^ ° d^ 
sgbsample of e x pe r 1 me n t e I s In Fairfax County. random 



NDTESs Performence Indicators tire defined e 
component or stetus. 



8 aver aaalgnad to a partlcuL 



8 r 



( conti nued ] 
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TABLE C.3 (cbhtrriuedj 

_ P^'"*■°^•"^nce IndlcBtors are calculated ao e percentege of the tbtet 
n"tv"' ^" J''^^"''^ — group In agancias tut Fa i rf . . . i„ Fa , r ax 

county, performance indicators are calculated ae a parcentags Of a random 

Fa"i rfL' L . ' "'"'^ "'•^ revia^e'd. The ..uUs from 

Fairfax County are .aiohted to equal the total nuSber of Fairfax e xpe ri .antal a 
In the reeaerch samptai ^ imoiitoiu 

."Dar.gietration is defined ee beini daregUtered according to ESP 

-onthe Poff^'"?' *° ''''''' poyn-ente at any time dUMng the nine 

■onthe follOBing random aaeiennent. 

Sanctioning ratae are defined ae refarrel for senctione; 

to pro.rem'^'tefr P^"^"" J ° o " i ^ "esed on employment that in reported 
to proflrem steff. Program placement dete will not be ueed to measore impects. 

Siatieticef '^l^'!V" ^PP'^^^ " d ffarences between welfare .tetuses. 

Statieticel eignificance levete are indiceted es: * = 10 percent- = 5 
percent; = 1 parcent. percani:, - 5 



=176- O^fi 



ERIC 



TABtE C;4 



VIRSINIA 

_ . ALL AFPC_: DISTRIBUtlON OF EXPERi MENTALS BY PROBRAMr 

WELFARE^ AND EMPLOYMENT STATUS IN THE NINTH MONTH AFTER RANDOM ASSIGNMENT 

[AUGUST 19B3--SEPTEMBER 18B4 SAMPLE] 



APPLICANTS 



Sta toB 


1 

Perti ci pent 


1 

Non-Parti ci pent 


1 

Tote L 


On Wetfare 


23.3 


14^0 


37.3 


Od re g1 ste red 
Emp t oy ed 
Not EBptoyed 


4.7 
2.3 
2.4 


5.3 
li.4 
3;9 


10.0 

3.7 
6.3 


Registered 
Empt oy ed 
Not Emp L 6y e d 


16.6 
5.3 
13.3 


8;7 
2.2 
6;5 


27 .3 
7.5 
19.8 


Off Welfare { Deregi Btered J 
Empt oy ed 
Not EBptDyed 


16.3 
11 .4 
6.8 


44^4 
24.9 
18^5 


62.7 
36 .3 
26.4 


Total 


41 .3 


58.4 


100.0 


Total Nosber of 
Appl 1 can 1 6 


357 


5oe 


857 




RECIPIENTS 






^ 1 
Sta tae 


, 

Perti c1 pant 

I — 


Non- Parti c1 pant 


_ 

Totel 


On Welfare 


55 .0 




19^5 


74.5 


Deregi Btered 
Eepl oy ed 
not cnployej 


10.1 
1.7 
8.4 


6^3 
0.9 
5.4 


2.6 
13.8 


Regl sta red 
Enp I oy ed 
Not EnplDyed 


44.9 
7.3 
37.6 


13.2 
1 .8 
11.4 


58.1 
9.1 
49 .0 


Off Welfare ( De regl ete ri d ] 
Enpl oy ed 
Not Employed 


14.3 
7.6 
6.7 


11.1 
4.5 
6.6 


25.4 
12.1 
13.3 


Total 


68.3 


30.6 


Idd.o 


Total Nun be r of 








Red pi ante 


887 


383 


1280 



( CO h ti nue d ) 
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TABLE C.4 (continued) 
ALL AFOC 



Status 


T _ 

Parti ci pant 


1 

Non-Parti ci pant 


1 

Total 


On iislfare 


42.3 


17.4 


59.7 


Oeragi ste red 
EmpL oy ad 
Not Enptoyed 


7.9 
1 .9 
6.0 


5.9 
1.1 
4.8 


13.8 
3.0 
10.8 


Re gi 6t e re d 
EoipL oy id 
Not EiapLbyid 


34.4 
6.5 
27 .9 


11 .5 
2.D 
9.5 


45 .9 
8.5 
37.4 


Ofir Wni fnrn fnMrnn^ ct'nmHl 
ui 1 ivoLiaic li/oroQiofcBrBu] 

EiDpL by ed 
Not En pL dyed 


15.9 
9.1 

6;8 


24.5 
12.7 
11.8 


40.4 
21 ^8 
18.6 


Total 


58.2 


41 .9 


100.0 


Total Nun be r 

of ExperinentaL 8 


1246 


893 


2139 



SOURCE: Calculations from the Virginia Autbnatad Client Information Syaten and 
from program activity data collected froia ceee file records of e rendom subsemple of 
e xpe ri ne n t al s in Feirfex County. 

NOTES: Per ti ci peti on is defined as ever essighed to eny component. 

^"Liice other tables. Individuals who were off weLfere by tht ninth 
month efter reridom eBsignment were considered to be de r egi st e r ed. In other teblest 
deregi stared is defined es being deregistered accdrdihg to ESP progreni records or 
felling to receive AFDC peyments at eny time during the nine months following Fendom 
a 6 si gnme n t . 



Due to the weighting of the Fei rf ex sample, simple sizes mey 
those of other tebles. 



not metch 



For enrottees rendom t y sbbI gne d durihg Octobir 1983, Jenueryi April, 

or July 1884, erapLpyment stetus in the ninth month is defined es ever empldyid during 
querter 3 (the second fot tow-up quarter]. _For the rest of the Simple, employment 
status Is defined es ever employed during querter 4. This procedure wis fdllowed 
beceuse the ninth month fot tow i ng random essi ghmerit f e I I s w i th i h the third querter 
for thi first group and during the fourth quarter for the rest of the simple. 



Tests of stetistical cignificertce between pirticipents and 
non-participants were not exenined. 
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TABt! GiS 
VIRGINIA 

TYPES OF JOB SEARCH ASSISTANCE AND REQdIREHENTS, BY AGENCY 







— Reqiilreiiient 


Agency 


Type of Job Search AsBiatance Provided 


Duration 


Noiiiber of 




Eiiiployer Cohtacta 


Fialrfax 


Group: Job Search Vorkahopa; Fottoied by 
_ loditidoal Job Search 
Individual Job Seerch 


2 ifeeks} 2 Weeks 
2 Weeks 


N/A; B 
n 


NeNport Ne«8 


Group PrientBtioniSeaaj one; Fottoiied by 
. lodhldDBt Job Search 
Individual Job Search 
Group Job Seerch tforkahope 
Ongoing Support Groups 


3 Half-Dav8i4 Weeks 
4 Weeks 
2 Weeks 
N/A 


n/ M 1 1 c 

12 

N/A 

N/A 


Hempton 


Group Job Search Rorkshopa 
Individual Job Search 


2 Weeks 
4 Weeks 


8 

ib 

1 u 


Chesepeeke 


Group Job Search Korkahops 
Individual Job Seerch 


2 Weeks 
4 Weeks 


12 


Henry 


Individual Job Search 
Group Job Seerch Iforkshops (During a 
One-Year Period] 


4 Ifeeka 
2 Weeks 


26 

N/A 


Gray eon 


Indivlduel Job Search 


4 Veeks 


8 


Gatax 


Indlvlduel Job Seerch 


4 Weeke 


16 


Carroll 


Individual Job Search 


4 Weeks 


12 


Mertlnevltte 


Individual Job Search 

^roup Job. Search fforkshops [During a 

One-Yeer Period) 


2 W8ei(8 


16 
N/A 


Pltteylvenle 


Indlyiduai Job Search . 
Group Job Saarch Vorkahops [During a 
Oni-Year Period) 


4 Weeks 
2 ifeeica 


12 

m 


Canpbell 


Individual Job Search 


4 Weeks 


12 



SOURCE! Intervleis with program staff. 
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APPENDIX D 

ANALYSIS OF PARTICIPA TION BY LOCAL .ftOENGY 



this Appendix expands the discussion of patterns of program partici- 
pation in Chapter 3 by examining variation among local agencies. As 
indicated in Chapter 1 , the ESP model established guidelines within which 
local agencies were allowed considerable leeway in implementing the 
program, 

I. ABsessnent for Assignment to Program Components 

Since ESP Worl:ers assessed enrol lees before assigning them to compdn- 
entSj the assessment process affected participation rates. In fact, the 
percentage of enrollees assessed at the different agencies varied from 63 
percent (at Newport News) to 100 percent (at Grayson). At every agency 
except Newport News, over three-quarters of the experimental s were 
assessed. 

The proportion of experimentais at each agency who were applicants at 
the time of random assignment could in part account for different assess- 
ment rates sinr^ applicants whose requests for welfare were not approved 
did not have to participate in or be assessed for ESP. As shown in Table 
2*3 f the proportion of applicants varied considerably across agencies. 
Difficulty In keep»ing track of enrollees who did not attend assessment 
interviews also influenced assessment rates. Some enrollees broke 
successive appointments, yet provided excuses considered reasonable and 
said they intended to adhere to program requirements. The pace at which 
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different agencies scheduled the on-board caseload for assessment 
interviews also might account for variation In assessment rates across 
agencies* 

the distinctly lower assessment rate found i'cr experimentais in 
Newport News (63 percent) may be due to the accelerated schedule for 
assessing the on-board recipient caseload of that agency. (The agency in 
Newport News did not differ markedly from the others included in the 
evaluation according to the other factors discussed above,) Since the 
staff in Newport News decided to assess the entire oh-board caseload in the 
first few months after random assignment began in August 1983, workers were 
faced with scheduling a large number of enrol lees in a short time — and 
might have found it particularly hard to keep track of enrol lees who missed 
appointments. In addition, during this period, new workers were being 
hired and taking over responsibility for enrollees from other staff 
members, thus raising the additional issue of possible confusion about the 
staff person responsible for tracking those who failed to keep their 
appointments. 

This Appendix refers to figures in Table D,1 which presents partici- 
pation rates calculated for all experimentais. However^ since the program 
did not serve those who were not assessed, rates were also calculated for 
those experimentais who were assessed within nine months after random 
assignment. Overall, 73 percent of the experimentais assessed were active 
in ESP, For most agencies, these recalculated rates do not represent 
substantial changes from the figures presented in Table D.1. Since the 
agency in Newport News assessed a smaller proportion of experimentais than 
any of the other agencies, Table D.2 does provide a more accurate view of 
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VIR6INIA 



ALL AR)C! NINE-WINTH PEflFDRlcj INDIDIT0R8 m ~mmm, BY LOCAL /fiBICY 
lAlBiiST 1983 - SEPTEffiER 1S84 SftKlE) 





Urban Agencies . 


Rural Asenclea 


1 


Pepfofiance Indicator 


Fal rf ax 


1 

Naiport 


1 

Hanptdh 


Cheaa- 
peeke 


Henry 


1 

Grejaon 


1 

Gelax 


1 

Carroll 


Hartlna- 
vllla 


1 

PUtBVl- 

van! a 


1 

Crnipbal . 


_ 

.ML 


AfiMBGed^ 


S4,0 


63.0 


80.9 


91.3 


78,0 


1OD.0 


86,4 


89.8 


78,3 


97.9 


79.1 


7g,7*»* 


Ever ActivB 


59,0 


43.4 


63.1 


B8.1 


88.5 


64.6 


63,6 


52.5 


SS.2 


60.6 


53.0 


5a.3»»» 


Participated Iri Job Search 
Individual Job Search 
Group Job Search Actlvltlea 


62.4 
50.6 
4.2 


35.4 
10.2 
29.3 


50.9 
45.0 
7.8 


63.5 
49.6 
25.1 


66.1 
61.0 
11.9 


78.9 
76.9 
0,0 


59.1 
59.1 
0.0 


49.2 
49.2 
0.0 


51.7 
41,4 
17.2 


77,1 
78.4 
4.2 


42.6 
42,6 
0.0 


51.0»*» 
40.4»*> 
14.7*«» 


Parti ci bated in ilnrk 

< HI itiviVQvDU III nUl n 


























Experience 


10.0 


6.0 


16.3 


15.5 


0.0 


11.5 


16.2 


5.1 


3.4 


2.8 


4,3 


B,5«* 


Participated In Training 


























or Education 
Training 

Education 


7.7 
3.3 
4,4 


14.5 
3.6 
11.1 


11.6 
3.4 
B.i 


11.1 
4.3 
6.8 


11.9 
B.5 
3,4 


11.5 
11.5 


22.7 
18.2 
4,5 


11.9 
5,1 
6.8 


10.3 
6.9 
3.4 


11.1 

s.e 

5.6 


10,4 
4.3 
8,1 


4,3»» 
7.4»* 


_ b 

Que to Sarctionlng 


5.5 
7,9 


26.1 
0.9 


41,9 
3,1 


51.7 
5.0 


45,8 
S,1 


46.2 
11.5 


59,1 
13.6 


49.2 
1,7 


69.0 
10.3 


28.9 
1.4 


37,4 
0,0 


42.3*« 

- 8 

3.B 


d 

Placement Rate 


0.9 


9,2 


18,8 


31.9 


28,8 


30,6 


31,8 


11,9 


10.3 


19.4 


9,6 


14.1 


Total Ninber of Experimental a | 


454 1 


5B7 


320 


3^ 


59 


26 


...22 


59 


26 


1U 


115 


2138 
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T%LE D.1 Icdntlnued] 



r 

H 

OD 
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SODRCEi ISlculitlonii ffci fHs Virginia AutoiiBEed Client Infornifttl on Syeteri and fri prbgrm activity date coUected from ceea flte 
racorde of e mim iubeiiptfl of experiientele lit Fairfax CdUntyi 

NOTESt Parf'dhilenca indrcatbrfi ere defined ae ever aaiigned to a particular ccipbhant or Btetue, 

_ftrfdNlSnc"e indicatora are celciileted as a percantege of the total parabna in the indicated reaearch group In att agenciee 
bot Fairfax. In Fatrfex Countyt parforriance Indicators are celculeted aa e percantege of e rendom aubaeinpla of 183 expertmantaiB iihoae ceae 
fllea were reviewedi The resuita from Fairfax County ere weighted to equal the total nuiiber of Fairfax axperleantata In the reeearch 

Aaeeeieht Is defined for appllcaritB and redBtemilnad fflNahdatbry reclplante ee contact with an Empioywent ServiceB Iforker 
after randbn eBelgnmenti For thOBa recipients who were WINendatory before the start of the reaeerch, essasaDent la defined bb 
reaBBeeaneht at the point of randoft aaslgnnienti 

'^DfiMfll^tffitlon Is defined as being daregletered eccording to ESP progran records or falling to racelva AFOC paynenta at any 
tine during the nine ibnths following rendbin esslgnmenti 

c • 
Sanctioning ratee are defined as referral for sanctlona. 

''progrw plBceiiiBnt inforfflatlon la based on employitient that la reported to program etaff. Progrm placaifient dete will not be 
Qsed to (tieasure 1ipecte« 

Chl-equBra teat Inappropriate due to liN expected cell frequenclesi 

A chl-equare teat was applied to differences among local egencleB. Statistical significance levala are Indlceted ee ♦ = 10 
percent; = 5 percent} s 1 percent, 



241 

FRIC 



T«tE D,2 



VIRGINIA 



NINE-fBNlH reRFCRfiffi IHOIttTORS FORJSSESSB) EXPERIffiNTALS, BV LOCAL «BICY 
(At)GDST19S-SEPrEffiER 1984 SAHPLE) 
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OD 
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. diten jUgsncjeB 








Rural Aeenci 
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1 

Newport 


1 


1 

ChBGa- 






1 


1 


1 

! 

llBrtin^ 


Plttayl- 


1 




PBrfoiTjnce Indicator 


p«{ rf ftv 

1 Q 1 1 1 HA 


Nmn 


Hfmntnni 

ilullipuVJil 


poo ISO 


Henry 


Oreyson 


Galax 


Carroll 


villa 


vanle 


Campbell 


Total 


Evor Active 


70.2 


68.9 


77.6 


74.6 


89.1 


84.6 


73,7 


c§-e 
5B|5 


69.6 


62.3 


67.0 


73.0*** 


Participated In Job Searcn 


62.4 


56.2 


62.5 


69.5 


84.8 


76.9 


68.4 


54.7 


65.2 


78.7 


53.6 


63.9*** 


Individual Job Sasrch 


60.2 


16.2 


55.2 


54.6 


78.3 


76.9 


68.4 


54.7 


52.2 


78.0 


53.8 


50.6**» 


- - - — — •• 

Group Jcb Search Activltiea 


5.0 


46.5 


9.7 


27.5 


15.2 


0.0 


0.0 


0.0 


21.7 


4.3 


0,0 




18.5*»* 


Participoted InMork 


























pyrin rinnnfl 


11.9 


9.5 


19.7 


16,9 


n n 
u.u 


1 1 .3 


21.1 


5.7 


4.d 


O 0 

c.t) 


Did 


11 ottt 


Parti cipe ted in Training 


























or Education 


9.2 


23.0 


14.3 


12.2 


15.2 


a.i 


26.3 


13.2 


13,0 


11.3 


13.2 


14.6**» 


Training 


4.0 


5.7 


4.2 


4.7 


10.9 


11.5 


21.1 


5.7 


8.7 


5.7 


5,5 


5.3* 


Education 


5.2 


17.6 


10.0 


7.5 


4.3 


11.5 


5.3 


7.5 


4.3 


5.7 


7.7 


9.3»»» 


.. __ a 

Deregi stored - 

b 

Due to eanctioring 


4B.9 


28.9 


34.0 


50.5 


32.6 


46.2 


57.9 


49.1 


65.2 


29.1 


33.0 


39,9J** 


7.1 


1.4 


3.9 


5.4 


6.5 


11.5 


15.8 


1.9 


13.0 


1.4 


0.0 


4,3 


--- -- -- c 

Ptacement Rate 


1.0 


14.6 


22.6 


34.9 


37.0 


30.8 


36.8 


13,2 


13,0 


19,9 


12.1 


17,7*** 




























Total Nnnber of 


























Aeeeseed Individoate 


381 


370 


259 


295 


46 


26 


19 


53 


23 


141 


91 
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*'^fl'"<> Cl.!*"* InforuBtlon Systai end fri ppogran actlyjty data cottectsd froi cau fItB 

rocordi of a randon iDbBsaipte of aipeplMntalB In Fairfax 



NOTES: PBrfonsnca liidlcatdrS lira defined es ever aeiigried to a partlciilep conponent or etatua, 

.^'^^i'"'"" calculalad aii a percentage of the total pereonB In the indicated raeiirch groqp lii ett egencl 

but Fairfax, In Fairfax Coantyi perforBianca Indlcatore are calculeteG' as a percentege of e random subeonpte of 183 experlnantata whoee 
fiiee were revleiad. The resatte from Fairfax County are ielghtad to equal the total nimber of Fairfax. expeflmentate In the reBaarch 



a 

DerBglitrBtlon la defl/iad ae being dereglBtered according to ESP prograti recorde or felling to receive AFDC pByiiientB at any 

time during the nine siontha fottoilhg random eael gnment, 

SBnctlShing rates are defined as referrel for sanctlona. 

0 

'^''°9''" Pl*ai»Bnt Infdriiiatlon Is baeed on ampioynant that is reported to prograi) staff, Progrnii ptaceieiil data lUl not be 

I used to DieBBgre iRpacte, 

i Chl-Kjuare t^iat Inappropriate due to loii expected cell frequencjes, 

... . . * ■PPH'"' ^° differences anong local agencleB, StatlBtlcal significance tevete are Indleeted bb » r lo 

percent; = 5 percentj = 1 percent, 



assignment patterns in Newport News, 



ii. PartiolD ation in Program Components 
Ai Job Search 

As was found in the interim report, roughiy similar proportions of 
experimentals from the 11 agencies participated in job search during the 
longer follow-up period. (See Table D.I.) Interviews with staff suggested 
that at all of the agencies, staff considered participation in some form of 
job search to be mandatory. Across agencies, individual job search was 
used more than group activities. Because of its flexibility and minimal 
requirements, individual job search was considered a reasonable assignment 
for all registrants without medical or mental health barriers to participa- 
tion. Registrants who were working part-time were also sometimes excused. 

With the exception of the three-day group orientation that preceded 
individual job search in Newport News, group job search activities took the 
form of workshops. Assignment to workshops was generally not mandatory; 
enrolleiss who did not wish to participate were often assigned to individual 
job search. 

In contrast to the early implementation period covered in the interim 
report^ three of the agencies serving rural areas were able to provide 
group job search services during the final year of the evaluation. From 
July 1984 through June I985, the agency in Martinsville operated job search 
workshops under a JTPA contract for a consortium of six local agencies, 
including two others included in this evaluation, Pittsylvania and Henry, 
these workshops lasted for two weeks; the first week was devoted to 
instruction^ while portions of the second provided practice in telephoning 
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employers. (See Chapter 3 for a description of the content of the job 
search services offered through workshops.) The site of the workshops 
rotated, with each agency served under the contract hosting approximately 
three workshops over the course of the contract year. If the host agency 
assigned fewer than about 15 participants to a workshop, then the other 
agencies were allowed to fill vacant slots with enrollees able to get to 
the workshops — either on their own or with the assistance of staff or ESP 
enrollees who had cars. 

In Pittsylvania, only a small proportion of experimentals were 
assigned to these workshops, but larger proportions participated In 
Martinsville and Henry County. (See Table D.I.) Since the rural areas did 
not have public transportation systems, the ability of the enrollees to get 
to the workshops often determined whether or not they were assigned to this 
activity. 

Staff had to expend a considerable amount of time to find appropriate 
facilities for the workshops, to make sure enrollees had transportation to 
them, and to assign an adequate number of enrollees to fill them. An 
example of the kind of situation which required considerable efrort on the 
part of staff was presented when the host agency did not have adequate 
facilities for telephoning potential employees; staff had to arrange to bus 
participants to the Martinsville agency for that part of the workshop. 

Although staff believed that workshops provided valuable assistance, 
few participants subsequently got jobs. Martinsville was unable to meet 
the placement standard in its JTPA contract (60 percent of all partici- 
pants) and was net awarded a job search contract for 1985. 
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B. Work- Experlerice 

The wide variation in the level of pa; tic.tpi:tici*i in work experience 
found in the interim report is also evident in :;c^;a now available for the 
full sample over a longer follow-up period. (See Table D. 1 . ) Mo experi- 
inehtals participated in work experience in Henry County, where local 
government officials were hot willing to create work experience positions 
in county agencies. In addition^ by the end of the period covered by the 
evaluation, staff in Henry County concentrated on developing referrals to 
education and training activities to prepare enrollees for the service 
market in the area. 

In contrast to Henry County, more than 15 percent of experimehtals 
were assigned to public and private nonprofit agencies in Hampton, 
Chesapeake and Galax, and the other agencies assigned a smaller proportion 
of enrollees. Since, according to the ESP model, work experience must 
follow job search, and, over time, enrollees leave the program for a 
variety of reasons, the highest rates shown in Table D. 1 represent 
substantial operational achievements. 

Several factors contributed to the observed variance in participation 
in work experience. As was the case for job search, the absence of public 
transportation systems influenced the low rates of participation in work 
experience in most of the rural areas. A primary concern of staff in 
making assignments to work experience was the ability to get to a worksite. 

Another factor that accounts for the pattern of participation in work 
experience was the initial lack of consensus among staff in most of the 
agencies about the value of work experience for a broad range of 
recipients. No clear criteria established who would most appropriately be 
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assigned. Desire to maintain good relations with employers In the 
community ialso influenced staff, who were concerned that enrollees Who had 
not undertaken the activity voluntarily Sight perform poorly on the work- 
sites. As the program was implemented, participating enrol It js and their 
supervisors responded positively to this ESP component, leading staff to 
begin to assign registrants who were hesitant about volunteering. This 
change occurred faster and was more extensive in agencies :?hich placed more 
emphasis on consistency in assignment policy than in those which allowed 
individual ESP Workers more leeway to act according to their own inclina- 
tions. Since ESP Workers exercised considerable discretion in all of the 
agencies, these assignment practices could vary among staff- within an 
agency as well as between agencies. 

III. Education and Training 

Rates of assignment to education and training services varied little 
among the agencies. To the extent that they did, the way in which they 
varied was contrary to the expected pattern. Especially because enrollment 
in education and training programs Was voluntary, it might have been 
sensitive to the availability of these resources, which were more limited 
in the rural areas and in one Urban site — Chesapeake — than in the other 
urban areas — Fairfax, Hampton and Newport News. Yet, experimentals in 
some of the rural areas Grayson ahd Galax — participated in training at 
higher rates than their counterparts in the urban areas. The rural 
enrollees from Grayson, as well as those from the more urban Newport News 
and Hampton, were more likely to have taken part in an education program 
than those in other agencies. Interviews with staff suggest significant 
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factors in explaining these rates of participation: strong encouragement 
that enrollees take advantage of appropriate opportunities as well as 
persistent, ongoing efforts to promote cooperative linkages with Job 
Training Partnership Act (JTPA) service providers and other education and 
training institutions. 

in summary, the extent of participation in post-job-search components 
depended on local factors such as the availability of public transportation 
and the degree of emphasis on developing relatively uniform assignment 
policies. 

IV. Deref^istration 

The data available for analysis do not permit full explanation of the 
variation in dereglstration. However, data are available for a potentially 
Important reason for dereglstration — sanctioning. People who did not 
comply with program requirements and did not provide adequate reasons for 
noncompliance were referred to their eligibility worker for sanctioning (or 
reduction of their grant). During the three-month period over which the 
grant of the sanctioned person was reduced, the enrol lee was deregistered 
from the program. Data on referrals for sanctioning indicate that, in most 
of the agencies, sanctioning did not constitute a major reason for deregls- 
tration. Staff reported that the majority of enrollees who at some point 
did not comply with ESP requirements either provided reasons considered 
adequate or responded to a warning of possible aanetloni 

Across agencies, the most frequent reason for no ncbmpi lance was 
failure to keep a succession of assessr?nt or reasssesment appointments. A 
minority of agency staff Indicated in interviews that enrollees were just 
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as likely to be referred for sanctioning because they failed to complete 
the ESP job search requirements. In interviews, staff at all the agencies 
reported that they sought to contact enrol lees who had not met program 
requirements and to devote time to resolving problems before recommending 
sanctions. Nonetheless, the sanctioning rates reported in table D. 1 do 
vary, reflecting differences in agency policy regarding the value of 
sanctions as a means of changing behavior, the legitimate reasons for 
imposing them, and discretion allowed individual staff members. 
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APPENDIX E 




APPENDIX E 
THE WORKSITE SPRVEY 



As indicated In Chapter 3, interviews with a random sample of i}7 
participants in ESP work experience and their supervisors investigated the 
types of jobs assigned to participants, the extent to which participants • 
skills and work habits improved, their level of satisfaction with their 
assignments, and their beliefs about the fairness of the work-for-benef its 
approach. These interviews were conducted between March 198^1 and February 
1985 with a standardized instrument that MDRC designed and is also using in 
its evaluation of the five other state programs in the Demonstration of 
State Work/Welfare Initiatives that include a work experience component. 
Besides Virginia, these programs operated in Arkansas, California, 
Illinois, Maryland and West Virginia. 

As indicated in the interim report, most of the jobs held by the 
survey participants were entry-levei. Almost ail were either clerical 
assignments (2g) or service positions (15), which often involve j serving 
food. Descriptions of typic^^, jobs To" low: 

• PSS Sierk: gives oul sut%.M^c% makes pfiotbcbpies, Stamps 

documents, collates mat .'^ial. 

• -Clerk/tVDist in t he xs^m^^ r^in^s i:.^:^?^^ ^ Veterans Admlni tstrg- 
tion h ospital ; keeps jctient a^i-^'^, ord^r^-s suppiiest /^ys 



bills. 



School -cafete ria helr > .-r-: 
wiches, serves food, sv^'^ei:s.. 



< i 3hec, 



.-Ips make ssnc!- 




. VAtvl^.-. takes patients 
tv:%^ti6n Center and 



-196- 



253 



• Nursing AssjrStaft^: makes bisds, transports patients to the 
X-ray Department^ takes the temperature and measures the blood 
pressure of patients. 

As shown in Table E,1, entry-level jobs such as these do not require 
many skills. Supervisors were asked to assess the importance of cognitive 
and general working skills for the jobs held by participants in work experi- 
ence. None of the cognitive skills (reading, writing and arithmetic 
abilities) was considered to be important for any of the jobs assigned to 
work experience participants in the sample. However, some of the general 
skills were judged valuable — with working well without supervision 
mentioned most by supervisors (by 20) and communicating well cited least 
(by only 3), fts for the importance of the ability to use a variety of 
tools, simple machines were mentioned most frequently (by 18 supervisorfi) , 
followed by complex machines (16), simple tools (10) and tools requiring 
dexterity (3). 

In Virginia, the number of skills required for work experience 
assignments was lower than in the other five states in MDRC's Demonstratibh 
of State Work/Welfare initiatives. In Arkansas — the state with the 
highest number of skills required for the assignments — communiGating well 
and cooperating with co-workers were cited for all the jobs. Even the 
general skill required the least -- using one's own initiative -- was 
important in nearly two-thirds of the assignments. 

For the limited skills that were required in Virginia, participants 
were generally already adequate when they began their assignments: in most 
cases, fewer than six participants were judged Inadequate, However, of the 
20 participants assigned to positions in which working without supervision 
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TABLE Ei^ 



VIRGINIA 

ADEQUACY OF WORK. EXPERIENCE PARTICIPANTS IN SEtECTED SKILLS AND WORK HABITS 
IMPORTANT FOR THEIR JOBS, AT THE START OF THEIR JOBS 
AND AT TIHE OF INTERVIEWS^ AS JUOSED BY THEIR WORKSITE SUPERVISORS 



T 



Tv oe of Ski t t oi^ Wsrk Hahi^ 


. . . _ 

Number of Work 

Expe r 1 ance 

Jobe Where 

Skill Is 

1 M n n 4* A n ^ 


Number of 



Adeq ue t e or 

More Than 

Adequate et 

Start of Work 

Experience Job 


Perti c1 pants Wh 

1 

Ine deq ue te e t 
Stert of Work 
Experience Job 


e 

0 Were: 
1 

_ 

Ine deq tie t e 
et 
Time of 
Int erv 1 ew 


CDafilCivii Skills 










Readi na/Wri nn 

noaui iJ^/ >fl 1 k| 11^ 


U 









Ar i tliBBt 1 c 


0 


— — 




II 












W W Hf 9m U 1 1 ■ W O w Q IV W ^ ^ 


Q 

_ o 


d 


b 


b 


Cooperate With Co^Workers 


1 7 


1 b 


1 


b 


Dea t \ii tt. Pub 1 1 c 


1 7 


1 o 


A 


1 


Use Own 3! hi €1 6 11 v e 


1 1 


o 


tz 
u 


1 


Work Without Supbk* vision 


2C 


Q 


1 1 


1 


Abltlty to Use Tool 8 










Si npt e Too 1 8 




6 


2 


b 


To^ts Raqulrlng Dexterity 


_? 


3 


0 


b 


Sliapte Hach1ne8 


18 


7 


11 


1 


Conplex Machihe8 


16 


4 


12 


1 


Work Habits 










^ t tendance 


N/A 


42 


5 


2 


Concentrates on Tesk 


N/A 


42 


5 




Works Quickly 


N/A 


34 


12 


2 


Follows Ihstructibhs 


N/A 


41 


3 




Cells In Sick 


N/A 


42 


4 


2 


Completes Tesks 


N/A 


43 


4 


1 


taarns Fron Mistakes 


N/A 


43 


4 


1 



SOURCE: Ihtervievs conducted by HDRC Field Reseerch steff with the worksite 
supervisors of a re rid dm sample of pertici pints In work experience jobs between 
March 19B4 and February 1965. 



NOTES: N/A Indicates not applicable beceuse ell supervisors were esked to 
rata the adequacy of the participant. 

s ~ " ~ 

A totel of 47 supervisors were Interviewed. Numbers ere besed only on 
those jobs where the supervisor Indicated thet the skill wes Importent. Due to e 
change In q uest 1 oh f ormety not all supervisors were esked ebout participant adequecy 
In all Inporteht stcllls. 

b 

Date from one Interview Is missing for this question. 
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was an important asset, 11 were considered unsatisfactory at the start of 
their assignments. Somewhat higher proportions initially had insufficient 
skills for jobs requiring ability to use machines il of the 18 parti- 
cipants assigned to positions in which use of simple machines was 
important, and 12 of the 16 in jobs involving complex machines. 

In addition to assessing work skills, supervisors were also asked to 
judge adequacy in seven work habits which are commonly assumed to apply to 
most jobs and work settings. For each of these work habits, with the 
exception of working quickly, fewer than six participants were considered 
inadequate. 

The opportunity for work experience participants to develop skills was 
limited because the assignments required few skills and because most parti- 
cipants were already adequate in the skills which were important. However, 
those participants who were initially less than adequate in a skill did 
meet accceptabie standards by the time of the interviews: no more than two 
participants remained inadequate In any skill or work habit; 

Despite the low level of skills required, the assignments were rrt 
"make-wdrkj" that is^ of no importance to the worksites or the participants 
assigned to them. When supervisors and participants were asked to choose 
from a series of statements describing the value of the work to the agency, 
most of the jobs (39) were described as **necessary work;" (See table E.2.) 

Participants* responses to the survey revealed a high level of job 
satisfaction. (See Table E.3.) Thirty- three of the participants strongly 
agreed that the work would lead to a job with decent wages and 30 strongly 
disagreed that they had learned nothing new on the job. Finding that this 
high a proportion of participants felt that they were gaining skills or 
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TABtE 



VIRGINIA 



CHARACTEniZATlDN BY WDRKSITi SUPERVISORS ANO PARTICIPANTS 
OF WORK EXPERIENCE JOBS IN TERMS OF IMPORTANCE TO THE AGENCY 



OsQreB or InportBncB 




numoer or rarcicipcnts 


Su-D^-r-V"i"8^-p-S-*— P^-TT* n n t i On 






NecBBBB ry Wor k 




39 


Work Can WBlt» But EventUBtLy 
Nefidfi bft bone 






HeLpB 1 f Work 1 s bone 




5 


Work is Not PBrticutarL^ 
InportBnt to AgBncy 




b 








NBCeaBBry Work 




39 


Work Can Waits Bit :i:vertuBLly 
NBeds to be dona 




2 


RaL ps 1 f Work is bona 




4 


Work is Not Parti cularLy 
Important to Agancy 




d 


ToteL Number of Sampladwork 
Exparlence Participants^ 


47 



SOURCE: IntarvieNS conducted by MORC F1 a t d Re see r c h Staff with 
a rendom sampLe of perticipents in «ork experience Jobs betiieen March 
1984 and February 1985 and thoir Morksita aapervlsore. 



ToteL does not edd to 47 due to missing response. 
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TABLE E.3 



VIRGINIA 

WORK EXPERIENCE PARTICIPANT RESPONSES TO QUESTIONS 
CONCERNING a DB SATISFACTION AT WORKSITES 



Que 8t i on 



OveretLi I L i ke ny job. 

Strongly Oieagree 

Soaewhat Oieagree 

Soirewhat Agree 

Strongly Agrae 

Generally apeeklngi hom do you feel most 
daye about coming to work here? In other 
wordst noBt days do yoas 

Look forward to coming to work? 

Not care one way or the other? 



Number of Participants 



2 

b 

10 

35 



40 

3 



Hata the 


thought of coming to work? 




2 


Whet abAut 


your supervisor and other 






reguler employees here — do you feel 






they Look 


on you SB pert of the reguler 






staff?® 






Yea 






40 


No 






1 


The kind of work I'm doing wilt help me 






to get a de ce nt- pay i ng job later. 






Strongly 


01 se g r ee 




3 


Sbmew hat 


01 sa g r ee 




2 


Sonaw ha t 


Agree 




9 


Strongly 


Agree 




33 


I ha^e nor 


leerned anything new on this 






J Ob. 








Strongly 


01 ea gr ee 




3D 


Sbmew he t 


01 6a g r ea 




4 


Somaw ha t 


Agree 




8 


Strongly 


Agree 




4 


Total Number of Work Experience 






Parti ci pa nta Interviewed 




47 



SOURCEi Intarviewe conducted by MORC F1«ld Riieearc^* Staff with 
e random sample of participante In work Bxpeiien::e j obe between March 
1SB4 and February 18B5i 



a 



NOTE: Oistribution does not add to 47 due to mlssinp 

ra 6pon ae a • 
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experience which would benefit them later is surprising^ gi\ in the gener- 
ally low level of skills required and the degree of adequacy in the skills 
that were considered important. One explanation for this finding may be 
that^ in contrast to the more concrete questions that supervisors were 
asked about specific skills, the general questions asked of the partici- 
pants tend to encourage overall positive responses. 

Hespon^i:es to three additional questions revealed overall satisfaction 
with the assit^nments. (See Table E.3v,) All but two of the participants in 
work experience said they liked the asr."s;r;^^n* ^, To each of the other tvo 
questions, HO participants responded ti>. : Ih^ looked forward to coming to 
^ork and felt the agency staff consider -^ tnem part of tvjf* regular st . . f . 

Most participants responded f^vcvabiy to working for wci fare benefits, 
(See Table E.i|.) Thirty-nine said they were satisfied to recelv? benefits 
tied to a job and E? indicated that they felt better about receiving 
welfare when they were working for it. Yet, 32 participants thought that 
given the usefulness of their work and the amount of their benefits, the 
work experience arrangement was better for the sponsors than for the 
participants; participants believed their work was worth more than the 
compensation implicitly at the minimum wage. (The required number of 
work hours was calculated by dividing the welfare grant by the minimum 
wage.) 

Participants were judged to be aware that the program was mandatory if 
they believed that their grants would be reduced if they did not meet the 
work requirement, either by refusing to take the job or by quitting it. 
According to this criterion, the great majority of work experience partici- 
pants were aware that although their preferences were taken into account in 
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TABLE 



VIRGINIA 

eURK EXPERIENCE PAPTjniPANT RESPONSES TO QUESTIONS 
CONCERNING Tftr FAIRNESS OF A WORK REOOIREMENT 
IN THE EMPLOYMENT SERVICES PROGRAM 



MusHbi on 


J 

Number of Perticipenta 


How latl if 1 ed are you about receiving 




wBLfari behefita Like this - that ia^ 




tbSd to a ioba inBi*. fiari nf cSmniv w» m f* t% 4 4 r% ^ 

J****! ui biinpLjr racBivinQ 




your be na f i 1 8 




Va ry Sati af i ad 


1 " 


Son B w het Saticf^fiH 


21 
3 


SODBwhst Di RRlit^ cf4nH 


iiMi* oabiDiiau ALL 


4 






now bnab JL an MorKing tor Ita 




Strongty Agree 


S3 


Somewhet Agree 


4 


n Am a Mt n A ^ O 4 a n m a a 

wunoivnok 1/ 1 BByres 


3 


S t r 0 h g L y OisagrlBjB 


14 


X * d L i ke to aek you how ueafuL your work 




is to the agency. Let's sey you :ggjB:P:a^Ma- 








SS noijei: you receive in benefits - who 




wdutd you aay prbbebLy Is getting the 




better and of *he deal: ^£jit or the 








Me 


e 


Neither One 


4 


Age ncy 


3S 


Dees pefticipent uhderatend thet pertici- 




— g 

pstion is Biehdetory? 




Yea 


34 


No 


4 


Totat Nainbef of Work Experience 
Parti ci pante Intervlawed 


47 



SOORCEi IhtarviewB conducted by MTRC Field Research staff with 
a random sampLe of participanta in work e perience j oba between Merch 
1884 and February 1885. 



NOTES Diatributlon doea not add to 47 due to missing 

responees. 



2C0 



making assignments to worksites, jsarticijsation in work experience was 
mandatory. 
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APPENDIX E - FQQTNQTES 



1i Besides Virgihiai the states in the Demonstration which included a 
work experience component in their work/welfare initiatives were: 
Arkansas^ California, iiiinois^ Maryland and West Virginia. For 
examples of findings oh the work experience component in those 
states, see,_ Friedlander et al,, 1985a; Friedlander et al., 1985b; 
Goldman et al. , 1985b; and Ball, I98i|, 
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TABti F.1 



VIRGINIA 



AFOC eONTROtSi EMPLOYMENT AND PROG RAM STATU S_ 
AT THE FbURtH QUARTER AFTiR RANDOM ASSIGNMENT 
(AUGUST 1883 - SEPTEMBER 1884 IMPACT SAMPLE) 



* f • t f ■ r_* Sti t u i 



Eaployavnt Status In Quarter Four 



Not EBptoyed _Ej^Pl oy ad 



Total 



AFDC Apptlcantt [%] 

Ravar f^acaivad ialfare In Qoartars 2-4 

Evar Ricilvad ifal '^rU In Quartars 2-4 
Off Wetfira by C-artar 4 
Still Racelvlnfi ialfare In Quarter 4 

Total 

AFOC Reel pi ahta [%] 

Never Received Welfare In Quartera 2-4 

^y^r l^acelvad Welfare In Quarters 2-4 
O*"*" Welfare by Quarter 4 
StU t Receiving Wolfere In Querter 4 



Total 

All AFOC [%] 

Niver Hbuciived Welfare In Quartere 2-4 

Ever Received Welfare In Quarters 2-4 
Off Welfare by Quarter 4 
Still Rnceiv^hg Welfere in Querter 4 

Toti t 



17.8 



10.8 
31 .8 

60.4 



4.1 



8.9 
62.8 

75. B 



8.8 



9.7 
50.1 

69.6 



17.7 



11.8 
10.1 

39.6 



3.2 



8.1 
11.9 

24.2 



8.1 



10.2 
11.2 

30.5 



35. S 



22 .6 
41 .9 

100.0 



7.3 



18.0 
74.7 

100.0 



16.9 



18.8 
61 .3 

100.0 



6DURCEX MORC calcuLationa froB Coaaonvbatth of Virginia UneBployBant Inaurenci 
airhihgs ricordet welfare racorda from the Virginia AutOBated Client Informetion 
SyitiDi ihd Fairfax County AFDC ca ia filaa. 



NOTES: These data are ragraaii on-idj uitad uiing ordiTiary lia£t aqueresi 
cbhtrolling for pra-randOB aapignBAnt cha re c t a r i a t1 c e of a&Bple Baitibers. Cue to 
rbuhdinsi there iay be aoae di scrapanci aa in calculating auas and differences. 

Monthly welfare detet which cbunt the sonth of rendon essignoient es 
'*Bonth ona.** ware ragroQped Into catahder quarters to aatch UI eerhihgs quertere. 
Percentages receiving welfere therefore will not natch precisely those in other test 
tables ("Querter 2") in the next querter after the querter of random assignment. 

Tests of steti nti CO I ai gni f i cance between those empLoyed end those not 
■inployed at Querter 4 were not examined. 
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VIRGINIA 



ALL AFDC: IMPACrrs OF THE EMft-OY^M^t SEfWieES raOGRAM. -B^'-TREATKCNT 
(AUGUST 1883 - SEPfEfeER 1984 IMPACT SAMPLE] 





ALL ESP 






Job Seerch 
¥/ork Experii 






Outcome and FoUoi-Up Quarter 


Services 


Contrci^B — Difference 


3nce Controls 


Oiff ersDce 


Evar BiplDyed^ Quarters 
2-4 (X}' 


43.8 


40.5 


+ 3.2* 


43.9 


40 .5 


+ 3.4* 


AveraSB NLinbfir of* QuArfcnrc W^lfih 

BBpLoymentr Quarters 2-4 


D.96 


0.65 


+ 0.12** 


0.91 


0.85 


+ 0.07 


Ever GnpLoyed (X] 

^r^^r of Random AsBlgrenent 
Quarter 2 
Quarter 3 
Quarter 4 


26.4 
28.6 
32.7 
35,2 


cO ./ 

26.4 
27.9 

3b i5 


+ 0.7 
+ 2.2 
+ 4.8**» 
+ 4.7V* 


26.0 
28.1 
29.8 


25.7 
26.4 
27.9 

oLJ.O 


+ 2.*^ 
+ 1.7 
+ 1.9 
+ 3.0 


Average Total Earningst 
Qaerters 2-4 ($] 


1127.34 


1036.15 


H^9.13 


1110.65 


1038.16 


+72.50 


Average Total Earnings 

Quarter of Random Assignment 

Qns r ^.ii r P 

Quarter 3 
Quarter 4 


217.38 

387.46 
_ 460. go 


224.22 
284.65 
346.14 
- 407.37 


+ 6.84 
+ 5.26 
+41 .32 
+63.12 


225.25 
290. ID 
373,75 
446.81 


224.22 
284.65 
346.14 
407.37 


+ 1.03 
+ 5.45 
+27.61 
439^44 


Ever Heceived Any AFDC 
Paynentsy Querters 1 - 4 (X] 


85.4 


86.1 


- 0.7 


86.6 


86.1 


+ 0.5 


Average Number of Months 
Receiving- AFDC Peymehts, 
Quarters 2-4 


7.66 


7 - go 


— n OA 


7. 85 


7.90 


- 0.04 


EyerReceivad Any AFDC 
Payment (X)- _ 

Quarter of Random Assignment 

Querter 2 

Quarter 3 


62.1 
76.1 
65.1 
58.8 


62.9 
76.4 
67.5 
59.6 


- Q;8 

- 0.3 

- 2.3 

- 1.0 


63.4 
76.6 
66.7 

60,6 _ 


62.9 
76.4 
67.5 
59.6 


+ 0.5 
+ 0.2 
- D.8 
+ 0.8 


Ayere^e TotBL AFDC Peyments/ 
Received^ Quarters 1 - 4 ($] 


1869.58 


2006.90 


-117.33** 


1957.47 


2006.90 


- 49.43 


Averege AFDC Peyments 

farter of Random Assignment 
Quarter 2 
Quarter 3 
Quarter 4 


533.56 
515.61 
438.04 
402,37 


551 .48 
546.72 
478.32 
430^8 


- 17.92 

- 31.11** 

- 4Q;28*** 
~ gfl.DI* 


551 .03 
529.54 
456.00 
418.90 


551 .48 
546.72 
478.32 
430.38 


- 0.45 

- 17.18 
~ 20.32 

- 11.4B 


SamptG Size 


1077 


1044 




106i 


1044 





(conti nued] 
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Ti^LE F.2 icbntlnued) 



H?^^ "^^^^^55^°?^^ ComSdhireBLth of Virginie Unemployment Insurance eorninoe recorder 
weLfere recordB frm the Virginia Autaieted Client Infonnetion System^ end Fei rfex County AFDC case fites, 

HOTESs Thaae d^te include zero values for sanpte iriflnbers hot anployed and for sonpLe m«ibers_not 

racai¥Jna:welfare. JIibbb data are reflreseidh-acyuBted using ordi nery_ Leest squaresi control lino for 
pra~randoo assigrsent characteristics of sample manbers. Due to rounding* there mey be ecxne diecrepancies 
in calculating auns and differances. 

- ^ -- The quarter of randoo aasignaaht refers to the celerider quarter during which ani ndiyl dual 
entered the BMspte for impldyBent end earnings data. For AFDC payments* the quarter of rendom eeslgment 
rafere td the three mbnths beginning with the month of randan asslyisient. 

a :: 1 _ _ 

Ouerter 1* the qaartar of rahdom assighmentr «ey cbhtiaii n abme earriinge fTCW.the.peripd prior 

to randcB aasigraant and Is therefore excluded from naaaures of total folloir-up for anptoyment end 
earnings. 

-^----^ , A^two-tBned^t--teBt was applied to experiraehtel-oDhtrbl differences. StetisticeL significence 

tevete ere indicated as: • = 10 percent; = 5 percent; = 1 percent. 

Differences bctjweBn lirtpacts were not significent et the 10 percent level using e two-teiled 



t-test. 
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rWLB F.3 
VIH:INIA 

AFDC APPCICANTS: IMPACfS-OF-^E EMPLPYfCNT SERVICES FflOGRAM, 

Bt TREATMB^T 
(AIK5UST 1983 - SEPTEWPfl 1984 IMPACT SAMPtEJ 



Outcome and FoLLow-Up Quarter 


AIL ESP 

Serv^ice^ 


C/A' V<2 Is 


_0i f f erencB 


Job Search _ 
_WDH<-Ej(peTl ence Control s 


Dif f iarerice 


Ever &optayBd| QiiartBr& 






4^ A 7 


60.2 


54 ;i 


+ o ,1 * 


Average NUQ be r of Month b Hjth 
EBptoymentt Quarters 2-4 


l;36 


i;i7 


+ 0.1 8»* 


1.28 


1.17 


+ 0.11 


EvBr_iiploy|Bd I%] _ 

Quarter of Random AsBionment 
Quarter 2 
Quarter 3 
Quarter 4 


44 ;5 
41.7 
46.2 
47.9 


47.1 
39.3 
37.8 

39^6 


- 2.6 
+ 2.4 
+ 8.3*** 
+ 6i3** _ 


46;9 
40.9 
41 .6 

_45i7 


47.1 
39.3 
37.8 


- 0.2 
+ 1.5 
+ 3.8 
+ 6.0* 


Average Totat Eern^ngBr 
QuarterB 2 - 4 [$] 


1606.35 


1504.58 


+101 .77 


1659.16 


1504.58 


+154i58 


Average Totet Eerninge [$] 
Quarter of Random Assignment 
Quarter 2 
Quarter 3 
£UiBi^tfir 4 


397.76 
408.36 
558.61 

^&iaa_ 


443.33 
452.60 
492.70 
-55S.28 


- 45.58 

- 44.24 
+ 65.91 
+ 60.09 


388.19 
439.12 

565^39 
654.65 


443.33 
452.60 

559;29 


- 55.14 

- 13.48 
+ 72.69 
+ 95.36 


Ever Received Any AFDC 
PaynentBi Quartars 1 - 4 [X] 


69.5 


69.2 


+ 0i3 


69.1 


69.2 


- Oil 


Avsrage Num^r of Months 
Racaiviho AFDC PavDehtB' 
Quarters 1-4 


5.00 


5.10 


- 0.1D 


4^91 


5.10 


- Oils 


Ever Received Any AFDC 
Psy*«nt ixl 

Quarter of Random AsBignment 

Q arter 2 

Quarter 3 

Quarter 4 


61.6 
59 ;2 
45.4 
38.6 


61.3 
56.2 
45.7 
40.2 


+ QiZ 
+ 3.0 

- 0.4 

- 1 .5 


61^9 
58.0 
44.0 
36.9 


61^3 
56 ;2 
45.7 
40.2 


+ 0.6 
1.g 
- 1.7 
-3.2 


AveraQe Total AFDC PaymentB 
Recelvedi Quartars 1 - 4 [$] 


1204^05 


1277.67 


- 73i62 


1200.75 


1277.67 


- 76.92 


I 

Ayeraje^AFDC Payments 
Received [$j 

Quarter of Random Assignment 

pMirter 2 

Quarter 3 

Quarter 4 


311.78 
368.80 
277.13 

246^.^33 


314.51 
384.69 
307.77 
270.70 


- 2.73 

- 15.Bb 

- 30.64 

-^-24i37 


320^07 
370.75 
272^53 
237.40 


314.51 
384.69 
3C37.77 
270.70 


+ 5.56 

- 13.94 

- 35i24 
--33.30 


Sample Size 


443 


428 




414 


428 





SOURCE NOTES s See Table F.2. 
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TABLE F.4 
VIRGINIA 



AFDC RECIPIENTS: XW»ACTS_0F_THE EMPLdY>€NT SERVICES ffiOGRAMr BY TREATMENT 
[AlCUST 1983 SEFTE*ER 1984 IMPACT SAMPLE) 



Oijtcbibe and FbllOMr-Up Quarter 


1 ALL ESP 
Serv Ice^t 


Controb s 


bif ferance 


Job Search 

Work-^xpej^l ehce CorvtT^s 


Difference 


Ever EnptDyedv Qoarters 

2-4 i%r 


33.7 


31.1 


+ 2.6 


33.0 


O 1 . 1 


+ 1.9 


Averaj.a Nuiber of Months W^th 
Eapto^nenti Qoartere 2-4 


0.70 


0.62 


4 0.08 


0.67 


0.62 


+ 0.04 


Ever Eihployad l%] 

O^iorter of Random Asaionnent 

Quarter 3 
Quarter 4 


14.2 
la. a 
23.7 
26.7 


1/ .4 
20.9 
24.2 


4 2.9* 
4 2.5 
4 2.8 
+ 2.5 


15.2 
18.5 
21.9 
25.3 


11.3 
17.4 
20.9 
24.2 


+ 3.9** 

+ 2.1 
+ 1 .0 

-+-1.2 


QoarterE 2-4(9] 


811.82 


713.73 


4 88.18 


741.12 


713.73 


+ 27.39 


erase Iota I iarnj nge [ IJ 

Quarter of Random Aseignment 
Quarter 2 
Quarter 3 
Quarter 4 


94.60 
195.95 
275.59 
34(L37 


-77.53 
168.40 
241 .83 
303.51 


4 17.06 
4 27.56 
4 33.77 
4_36.B6 


112.52 
187 .77 
245.90 

^QLi45 


77.53 
168.40 
241.63 
303.51 


•I- 34.98* 
+ 19.37 
+ 4^08 
f 3.84 


Ever Recaiued Any AFDC 
raynonbSf uuiorbero i iaj 


aO ml 


a/ .o 




98.4 


87.8 


+ 0.6yyy 


Average Nusber of Mentha 
Recei vlng AFDC Peyinentei 
Quartera 1-4 


9.44 


9.81 


^ 0.37*'^ 


8.83 


9.81 


- DiOlyy 


Ever Rocaived any AFDC 
Payment [X]- _ 

Quarter of Random Aaaignoient 

Quarter 2 

Quarter 3 


85.9 
87.3 
"^8.3 
72.4 


87.6 
80.3 
^.4 
73.2 


- 1.7* 

- 2i9* 

- 4.2* 

- 0.8 


87.9 
69.1 
81.9 
76.4 


87.6 
90.3 

82.4 
73.2 


+ b.3yy 

-i .c 

- 0.5y 
+ 3.2y 


Averege Total AFDC Payment e 














Recei vedy Qua r tare 1-4 ((] 


2351 .69 


2508.92 


-157.23*** 


2462.66 


2508.92 


- 46.26y 


Average AFDC Peymehts 
Raceiyed (1) 

Quarter of Random Assignment 

Quarter 2 

Quarter 3 

Quarter 4 


684.31 
614.31 
545.86 
507.21 


712.85 
658.39 
596.36 
541 .32 


- 28.54**_ 

- 44. 08*** 

- 50.50*** 

- 34.11* 


706.04 
635.81 
581.64 
539.16 


712;B5 
658.38 
596.36 
_ _ 541.32 


- -6.81y 

- 22.57 

- 14.78y 

2i16 _ 


SanpLa Size 


634 


616 


1 


647 


616 





(continued} 
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TABLE PiA [contfnoed] 



r.,.nJ^f^l: MDfiC^cBlcuiatiohs fr^ VirBiniB OnBmpLoymant Ineurance earhihge records, welfare 

r.coras fro. Virfllnia Auton.ted Cliant Infonnation System, and Fairfax County AFDC cised fillip 

Mc«i»fi?-Phfir!" 'net ude ^ero values f^^^^ and f^r Bsmpia gimbers not 

-«f,l^«S^^» : '■^B'""^^^^ "«'"B ordirary least squares, cbntroltinB for pre-randon-, 

l^s'^S diflarlnces. °' """" ' '"^ '""""^'"S' ^^''^ ^-^^ dlaorapanoies'i'„''cetcu'rat1ng°^^ 

■niDtM for i»hy!ifJi!t''f^!i ?^ '"f"*"jMi8nSent refers to the qwrtar durine which an individual entered the 
tt^Be «onthrh^^J^^?^„^i*r^s"*"^- ^^'^ P"y«8hEB, ttia quarter of randan assiB^nent refers to the 

three month B baginning with the nonth of randoBi BBBiflhineht. 111.01^1; 

a- 

r-nrtnr, =,^4.°?'^*^' - ' - BarhihoB frwi tha period prior to 

random asBignnant and is therefore excluded from mBBspree of total follOw-up for «,ploy«,ant and eern?nBs. 

l.v.1. are i ndicS^dllS"' Hrpe^HnSr-"' f plrSnlf ^^5'i"l"pBrca^^r'"""" ^"*^«'-^ «10h1f i..nce 

BiGnific-„r„ ii!7r^!"^^f^#*!^^ appUed to differences in inpects between treatments. BtBtisticat 
significance lovels are indicated bes y = 10 percent; yy = 5 percent; yyy = 1 percent. 
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TABtE F;'; 



VIRGINIA 



AFOC APPtiEANTS? IMPACTS ON EARNINGS .AND AFOC PAYMENTS 
IN PRaJECTION BASE PERIOd\ BY PERIOD OF RANDOM ASSIGNMENT 
(AOGOST 1883 - SEPTEMBER 1984 IMPACT SAMPLE) 



Diitcbinee for Last Two 
Follow-up Quartars 



Average Earnlnge ($j 

Average AFDC 
Peymente [$) 



Saiple size 



Aogoit - 
jptsnl 
1 983 



-1 92.78 



-88.24 



88 



^uerter of Random ABBlgnnent 



October - 
December 
1983 



JehUB ry 
March 
1984 



April 
June 
1984 



271.93 



-85,98 



318.99 



-72.60 



167.74 



-11.58 



345 



331 



276 



July - 
September 
1984 



205.71 



27.99 



265 



Full 
Sanipte 



223.43 



-43.17 



1285 



SOURCEj HDRC calcutatione from Coniohw-Bet th of Virginia welfare and Unenptbyment Ineuronco 
earnings records. 



NOTE!^! Projection Base Period refers to the taef two quarters of AFDC paymante and 
Unemployment Inauranca eernlnge data Bvaltabte for each Individual. tMb rafere to Quartera 7 and B 
for earninge and 8 and 9 for AFDC payments for those rehdoiiily asslgr between August end Septamber 
1983; Quarters 8 and 7 for eernlnge and 7 and 8 for AFDC paynentB fo; those randomly esslgned between 
October and Dacenber 1 983; Quarters 5 and. 6 for earnings ar,d 8 end 7 for AFOC peyments for those 
randomly assigned between January and March 1984; Qoartors 4 end 5 for eernlhgs arid 5 and B for AFDC 
peyments for thoBB randomly assigned between April end June 1984; end Quertars 3 and 4 for earninge 
and 4 and 5 for AFDC payments for thoao randomly assigned between July and September 1984. 

These date include zero values for saniple liiembers hot employed and for sample membere 
not receiving welfare. These data are regreseion-BdJ osted oslhg ordinary least aqueres, controlling 
for pre-randbra asslgriment characteristi ce of sample members. 

None of the dlffersncas between exporinientel and control groups were found to be 

Btatistically signiflcint at the 10 percent level osirig e two-tailed t-test. 

ERIC r-.:i 



TABLE F.i 



VIRGINIA 

AFDC RECIPlENTSi IMPACTS ON EARNINGS AND AFOC PAYHENTS 
1..' PROJECTION BASE PERIOo'i BY PERIOD OF RANOOH ASSIGNMENT 



(AUGUST 19B3 - SEPTEMBER 1984 IMPACT SAHPtE) 


Oiitcoraefi for Last Two 
FoUoif-up Ouartere 


Quarter of Random Aeolgnment 


\ 


AugoBt - October - danoBry - April - July - 
September December March June September 
1 983 1 983 1 984 1 984 1 984 


Full 
Semple 


Average Earnings (ij 

Average AFOC 
Paymente ($) 


161.23 158.84 154. 9e -67.75 -74.86 
-108,41 0.99 -19,13 -18.03 2.3P 


117.24 
-24.12 


Sanpla Size 


309 773 47S 232 108 


1B97 



SOURCE AND NOTES: See Table F.5. 
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TABLe F.7 



VIRGINIA 

AFOC APPtlCANTS: ESTIMATED REGRESSION COEFFiCIENtS E.IR TWELVE-MONTH FOttOW-UP, 

SELECTED EMPLOYMENT ANO WELFARE Me.AJURES 
(AUGUST 1 983 - SEPTIEMBER 1 984 IMPACT SAMPLE) 



VbpI able 


Ever Em p L by ied 
In Q ua r ter 4 


Ea r n i nge ^ n 
Quarter 4 


AFOC Rece i pt 
in Q ue r te r 4 


Total AFOC 
PayraentB in 
Quarter 4 


Expo r 1 ne nt a t 


+ 


+ 87 .47 


- 2.4 


- 28 ;6 9 


Group Mbmbar 


(2.8) 


(55.23) 


[2.8] 


(20.42) 


Rural 


- 3 .5 


- 67.23 


+ 5.1 


- 1 7 .4^3 




(3.4) 


155.91 ) 


[3.4) 


124.37) 


Aga 1 9-24 Yaara 


+ 1 B .2 


+2B8.52 


- 5.9 


- 63i90 




(14.5) 


(2B4o84) 


(14.5) 


(105i32j 


Age 25-34 Yaars 


•^^26. 2 


+500 .B5» 


-24„7* 


-21 4 ;28** 




(13.8) 


(271 .90) 


(13.8) 


(100;54) 


Aga 35 Years or More 


+ 21 .8 


-^535 .25* 


-24 ,Z* 


-221 i53*» 




(13.91 


(27 3.91 ) 


(13.9) 


(101 ;2B) 


Non-White 


- 2.3 


- 99.72* 


+10 .9*** 


+ 62i46*** 




(3.0 ) 


(58.07) 


(3.0) 


(21 .47) 


High School Diplome 


+10.1»»» 


+142. SB*** 


- 2.5 


- 14.49 


or Equivalent 


(2.7) 


(53 .81 ) 


(2.7) 


(1 9.90) 


Harrledi Widowed or 


- 2.1 


-117.77* 


- 4.i9 


- 63i89** 


D 1 V or ced 


(3.4) 


(67.67) 


(3.4) 


(25.02) 


Number of Children 


+ 2.5 


+ 5.61 


+ 8.0*** 


+ 88i86**» 


1.888 than 18 Years Old 


(3.1 ) 


(60.69) 


(3.1 ) 


(22.44) 


One Child 


- 6.0 


-230.75 


+15.9** 


+116. 34** 




(7.7) 


[151 .66) 


(7.7) 


(56.08) 


Two Children 


- 7.3 


-129. 26 


+10.4* 


+ 88i46»* 




(5.4) 


(1C6.22) 


(5.4) 


(39.28) 


On AFDC T»o Yaere or 


- 4 • 


-137.1 5** 


+ 5.B 


+ 37i66 


Le66 


(3.t j 


(69.88) 


(3.6) 


(25.84) 


On AFDS More Than Two 


- 4.5 


-186.60*** 


+17 .3*** 


+ 99il6*** 


Yea re 


(3.5) 


(66.10) 


(3.5) 


(25.18) 



[continued) 
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TABLi; F.7 (continued) 



I 



Variable 



Held a Job st Any Time 
During Four Quartare 
Prior to Randorn 
Aial gnnant 

^^rDl"96 Greater than 
$3000 In Four iQuarterr: 
Pr i or to Random 
A SB 1 ghm f - t 



Ever impt oy ed 
In D 08 r te r 4 



+1 B.3*** 

(3.3] 



E e . r h 1 h g 8 In 
Quarter 4 



+1 2.9*** 

(3.5) 



+249. 39*** 
(B4.01 ) 



+4B5.21*** 
(B9.34) 



AFUu Hecelpt 
In Que r te r 4 



• 1 .7 

(3.3) 



■ 4.6 

(3.s: 



Total AFDC 
Pay me nt 6 In 
Qua r t er 4 



+ 20^66 
(23.67) 



" 37.?(5 
(25.6^1 



SOURCE: MDRC calculatlbne from Commonwealth of Vlrglnle Unemployment i;.8urence 
earnings r eco r de ^ w a I f e r e recorde from the Vlrglnle Autometed Client Informetlon 
SyjBtefn^ and Fairfax County AFDC caee files. 



NOTES: The expe r 1 me nt e I b bre comprleed of both Job Seerch-l/n-k Experience and 
oil ESP Services reeeerch groups. 



and 



Sample elzes for the Aj-'dC Apptlcente ere as follows: B57 Expe r 1 mente I s 
n t r o I s . 



Regressions present 
presented In Teble 4,3. 



nhls tebte correspond Zc impect estlmetee 



^Btlmeted standnr^ errore ore *, i;^ parentheses, 
significance: = ld percent; = 5 percent; = 1 percent. 



el 6 of ete 1 1 St 1 ce I 



These data include zero vetues for eemple memberr: not employed end for 
sesipla members not receiving welfere. aU regression estlmetee ere by ordlnery least 



sq ua r es. 



EKLC 
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tA8LE i-.8 



VIRGINIA 



AFDC RECIPIENTS: ESTIMATED hESRESSlON COEF F=I CI ENTS FOR TWEtVE-MONTH FOLLO'^-UP, 

SELECTED EMFLDYMENT AND WELFARE MEASbRES 
(AUGUST 1933 - SEPTEMBER 1984 IMPACT SAMPLE) 



Varlsbte 


1 

Ev a r Enp I oy ed 
in Q UB r te r 4 


1 

1 

1 

Ea rni nge i n 
iuartar 4 


AFDC Receipt 
In Quarter 4 


t - 

Tofhi acnr 
luuQi, Aruu 

pay tie ht s 1 n 

Qua r t e r 4 


Expe ri nente t 


+ 1 .8 


+ .?d.18 


+ 1.2 


- 17 .92 


Groop Member 


[2.0) 


(38.21 ) 


(2.1) 


f 17.51 ) 


R 0 r a !; 




- 70.52 


+ 5.6*» 


- 49.7I5** 




{L.4) 


(45.86) 


(2,5) 


(21 .01 } 


Age 19^24 Yoar-B 


+ B.7 


- 63.33 


+18.3 


-146 .1 C 




[11 1) 


(2h9.25) 


(11.4) 


[95.80] 


Age 25-34 Years 


+ 3.7 


- 45.20 


+21 . 4* 


+1 89.1 9»* 




[ID. 6 J 


[199.643 


(10.9) 


( 91 ,48) 


Age 3 3 Ye ^rR or Mere 


- 1.5 


-i22.15 


+ 20. 1» 


+1 59.72* 




[10.6) 


t ; aGo79j 


[10.9] 


; 91 .55) 


Non-Wh i te 


+ 0.7 


- 10.95 




+ 62. £3^^* 




[2.SJ 


(43.56) 


[2.4] 


(1 9oSl. ; 


High Sciioot OipLome or 


+ 7 .8*** 


+107.13*** 


- 3.1 


- 26.74 


or Eq u1 J L eht 


[2.0) 


(37.^96) 


(2-1 ) 


(17.40) 


Merrfedt Widowed or 


+ 3.0 


+ 76. 03* 


-lb.4*»» 


- 88c07»** 


Divorced 


[2.3) 


[42 .83) 


(2.3) 


(1 9.63) 


Namber of Children 


- 3.8** 


- 48.85 


+ 3.1 


+ 97. 75*** 


Less then IB Years Did 


[1.9) 


(35.27) 


(1.9) 


(16.16) 


One \^nitd 


-10. 3** 


- 52.84 


- 2.2 


- 24.76 




[5.2) 


[ 97 .56) 


(5.3) 


(44.71 ) 


Two Chltdren 


- 8.9*» 


- 57.82 


+ 2.1 


+ 6.09 




[3.7) 


(70.31 ) 


[3.8] 


[32 .22 ) 


On AFDC Two Years or 


+ 10.2 


+203 .27 


+ 4.9 


-1 54.54» 


Less 


[9.5) 


(178.85) 


(9.7) 


(81 .55) 


On AFDC More Than Two 


+ 8.3 


+ 135.1 5 


+ 13.1 


- 85.25 


Years 


[9.4) 


(176.85) 


(9.6) 


[ 81 .04) 



( con t1 naed j 
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TABLc F.B (continued] 



T" 



Vr r1 abl e 



Held a Job at Any Tine 
DuMhg Four Quarters 
Prior to Random 
AbbI ghm ent 

^■fDl"08 Great or then 

$3DDD in Four Quartare 

Prior to Random 
a 

AsBi gnm er t 



Evar Bmpi oy ed 
in Q OB r ter 4 



+25 .9*** 

(2.5} 



+ 11 .3** 

(5.7) 



Ea rh i hge i n 

Que r t er C 



1. 



(46 .34) 



+767 .5q*** 
(107 .09) 



AFDC Rece 1 pt 
in Quarter 4 



•11 .6<»** 
(2.4) 



"15.5**» 

(5. 81 



Totet AFDC 
Peymente In 
Quarter 4 



(21 .23) 



33 iSO* 
t49.07) 



SOURCE: MDRC c a L c u La t i o n e_f r om CommonweeLth of Vtrgin'e Unemployment Ineurance 
eernlnge recorde, welfare pecord^ from the Virginie Autometed Client Informetloh 
System, and Fairfax County AFDC case files. 



NOTES: ThB exp e r i ir.e nta t s are comprised of both Job Search-Work Experience and 
all ESP Services raeaarch groups. 



and R16 Control 



Sample sizes for the AFDC Rec'plents are as follows: 1281 Expe r 1 rr e nt e 1 1 



Rearessions presented in thiB table cdrr^ipond to impact estimetes 
presented in Table 4.5. 



^^^^"^«^ed standerd orrprs are i n pa renth ebe s. Levels of stetlstice' 
Bignificence: • 10 ^jercent; = 5 percent; = i percent. 

These ud ta inclur^e zero vstuss for sample members not enpioyed end for 
sample nembers not receiving welfare. Alt regression estimetes ere fay ordinery leest 
sq ua r e 6. 
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TABLE G,1 



VIR6 INIA 

EXPERIMENTAL - CONTROL DIFFERENCES IN PROGRAM EFFECTS 
PER EXPERIMENTAL DURING THE OBSERVATION PERIOD, 
EXTRAPOLATION PERIOD, AND AT FIVE YEARS AFTER RANDOM ASSIGNMENT 



ESTIMATES POR APPLICANTS 





T 

! 

Obse rv a t1 on 


Pa r1 od 


I 




1 








Addl t1 onaL 
Fo t Loii-u p** 










Ootcone Varlabt^ 


Conibon 

----- a 
Fo I t b«-u p 


Ext r apol a 1 1 0 n 
Pe p 1 dd ^ 


Five Yaar 
To tat 


Ea rhi hgs 


• 127 


$216 


^1020 


to $1497 


$1363 


to $1840 


Fringe Banoflts 


23 


39 


1 84 


to 269 


246 


to 331 


Tax Paynents 


1 5 


26 


137 


to 201 


1 80 


to 244 


AFDC PeyiiahtB 


-7 5 


-70 


-1 93 


to -277 


-3 3 8 


to -422 


bthar TranBfer 

_8 

Pay Riant 6 


24 


-2 9 


-1 48 


to -21 0 


-1 5:- 


to -21 5 


Tra»n6fcp Program 
Adb 1n1 Gtrati on 


- 4 


-IS 


-6 5 


to -95 


-88 


to -116 



SOURCE: MDRC celcuLatlbhG frbra Uhemp^oyaient Insurance ciarhlhgG and peymehta 
r«corc3a; AFDC paynantG racordG; and pobtlshad and anpubtlshed date on cbx reteSr 
a»pLoyee fringe banefltCv and a dna 1 nl g t rfl 1 1 v e costG for AFDC* Hedlcsldi Food Stamps 
and UnenpL by Rent Insura'^ce, 



NOTES I Itt^ rk u*. -rs baGed on e sampte of 657 apptlcent experlmentets and A 
applicant c. fSy and era exprcGGftd in 1 984 dollarG, The dIfferenceG are 

ragressibn-tcf .>£tpd uG'tng brdlhary laest GquereG« cbhtrblling for pre-rendbpi 
fla^lghiheht characterlGtlca of ianpLr cemberG. 

*Th8 tonoon follow-up period coverG quertere 2 throug j for eerningcy 
fringe benefits end tax paynentGr end qudrterG 1 thrbugh 4 fbr AFDC peymentCf 
Medlcaldf Food StenpSr Urtan p t oy ne n t InGurence peym^jnCG end e dm 1 r1 g t r e 1 1 v e coGtG, 

b 

The additional fbllrw-up perlbd cbvere all the avelleble .61 Low- up 
quarters after quartiai* 4. The nuMber of addltloncl fblLOw-up 'i^uerters depends on 
each seDpLa Masber's ua ta of randon asaignnant, 

c~___ _"_ 

Eat 1 B8 te G raf I e ct alLirhatlve asaunptlbha abbut the change 1'. Impecte 
after dita cotCecton ended. The first nonbar f aech range asEomes that program 
affects decline by 22 percontpar yaar during the extrapolation period; the second 
nuBber est '""tc no dacey or increeee* 

^The five yaar totals are the sun of the estlneted effects d'.ir<ng the 
obGervetlon and axtrepoletron periods* 

e"___"__"_ - - 
IhcludeG Medlceld, Food Stenpe and Chetc- ^eht Inj^urance. 

^IncludeG adm 1 n1 Gt ra t1 v e coGtG for AFDT caldf Food Stempc and 

UnafflpLoynent Inaurence. 



TABLE G.2 



VIRGINIA 



EXPERlMENtAL - CONTROL DIFFERENCES IN PROGRAM EFFECTS 

PES EXPERIMENTAL DURING THE OBSERVATION PERIOD, 

EXTRAPOLATION PERIOD, AND At FIVE YEARS AFTER RANDOM ASSIGNMENT 

ESTIMATES FOR RECIPIENTS 





Obsa rv a t1 oh 


Re r < bd 






1 

i 






Outcoie Va rl ebi e 


C 0 ifl ■ 0 n 

Fo t tdw-up*^ 


Addl t1 ona| 
Foil bii-u p 


Extrspotatl Oh 
Pt r 1 od 


Five Ye 
Totat 




Ea rnl nga 




• 137 


$607 


to SB76 


iGD6 


to 


^1075 


Frnga Be he fits 


11 


25 


109 


to 1 SB 


145 


to 


164 


ta.: 'aymantE 


11 


21 


101 


to 1 46 




to 


178 


AFDC PaymentE 


-1 01 


-27 


-1 50 


tr -214 


-278 


to 


-3 42 


Other Transfer 

_B 

P«y nen t 6 


fi 


-1 9 




to -102 




to 


"113 


— - - (t 
Trancfer Prcgram j 

Adm 1 nl at re t1 op^ j 


- B 


i 


-2 0 




-26 


to 


-3 4 



SOURCE? MDRC calculattone from Unempt oyme Xneurer 

records; AFDC peymehtE recot^de; end pubUshed and unpub; 
employee fringe benefits, and ado 1 ni g t ra 1 1 ve coeta fct AT 
U hen pi by Be n t InEurence. 



and payme nt g 
on tex rates, 
^tcald^ Fooci StsnpE and 



^^"^^^i "^^^ f^^B^its ore baaed on e aenpLe of 1281 recipient expe ri me n te t c and 616 
recipient controls, and ere Qxpreaaed in 1984 doUers. The differences ere 
reoression-edjustcd using ordinery least equeree, cdhtrblUng for pre-rendom esslonment 
charecte ri Eti CE of sdmple nenbers. 

o-__ _____ 

Th3 common f o t I ow-o p pe r i od cb v e r s quarterE 2 through 4 for earnings, 

fringe benefits end tex peynents, and quertere 1 through 4 for AFDC payments. Medicaid, 

Food Stamps, Uneff.Jlbyment iHGurance payments and administrative costs. 

b : 

"The "ddi ti one I fot Uts-Qp pa ri Od cove,'6 c J. 4;he ovelL^ble follow-up 

quarters after querter 4. The nuaber of additional fott^N-up -ar depends on each 
Ganpte Denber's dato of rahdbs asaignipent. 

c : 

reflect atternatiwa assumptibha abbut the change in impacts 
The first number of a^ ch range assumes that prbgram 
effects cectine by 22 percent per yeer during the extrapolation period; the second 
nunbpp assumes no decoy or increase. 

d- 

^^ve year totals are the sura of the estimated effects during the 

observe tion end extrapblatibh peribde. 

e _ _ _ 

Includes Medicaid^ Food Stamps and Onem p t by men t Insurence. 

^^'^'^^"^es adraini strati ve costs t^r AFDC, Medicaid, Food Stemps end 
Unemployment Insurence, 
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ESTIMATED BENfcl^A?:- ftNi) ^I'TS^S PER EXFilfilME: 
AFTER FIVE /3 V PERSPETTIVE 



ESTIMATES FOf. ' E FULL SAMPLE 



Co Ib po h e h t of Anfilysis 



Incraasad Efirhings and 
Fringa Benefits 

Increased Tax Peynents 

Reduced Transfer 
Pey ine n t 6 

Reduced Trensfer 
Adm 1 n1 s t r e t1 o n 

Incransed In-Prbgre' 
Output 

ESP Operating Costs 

Support Service Costs 

Cbnnuhl ty Rdu CB t1 o n and 
Tral hi hg Cbets 



Pa r s pe ct 1 V I? 



Wet f e r 0 Samp l e 



Texpg y e r 



$1179 to l15Bi 
-149 to -2D1 

-390 to -499 



Nat Sain or Loss 



0 
?4 



to 

149 to 201 

390 to 499 

50 to 66 

104 
-3 88 
-24 

-1 8 



S o ci e ty 



$1179 to $1581 

d 



50 to 66 



1 04 



-3 88 
G 

-1 B 



664 to 5 



263 to 44b 



927 to 1 345 



SOURCE: MDRC caLcuLatlone fron VACIS and other progran activity bete; 

JTPA InforsBtloh eyetamB de ta ind flscel recordiy attanc^nce end flecet date 
froB local pubtl c achoots and cbmibuni ty cbllagaes MDRC «orksita survey; 
Unemployors.t Insurance ear.ilnge and peynehte recbrds; AFOC peymente records; 
ESP and repartient of Social Servlcae flacal records; MDRC study bf ESP etcff 
t1»e Btt&CBtlbh; a hci publlehed end unpublished de te on tex rotes, emptoyee 
fringe benefits, and adnl nl stre 1 1 ve costa for AFDC, Medlceldi Food Stomps, end 
Unaaploynant Inaurenco. 

wore eotliieted for the aene five yeer period es benefits 

elti.cugh nnst costo were Incurred In the first yeer ofter render, esslgnnient. 

Within eech peropectlve, positive numbers Indlcete gelns to tH-t 

group end nagetlve numbers Indlcete Looses. 

- Results ere expreosed 1 n 1 fl84 dollfirs. The totet semple 

Includes 2138 expo r 1 mente I o and 1044 controls; 

Po^bfibBfltsbtherthah In-program output, estlmetes reflect 
alternative assumptions about i^o change In Impects bftor dote collection 
ended. The first number of aech range assumes that program effects decline by 
22 P«''Coht per yeer during the ex t r epo I e 1 1 o n period; the second number esBumes 
no decay or Incraaee. 
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TABLE ei4 



ESMMATED NET GAIN OR LOSS PER EXPERIMiNTAt DDRING THE CiSSERVATlON PERIOO, 

BY WELFARE STATUS AND PERSPECTIVE 





1^ ' ' 

Pe re pec t1 ve 


Welfare Statue 


WeL fere Sample 


Budget 


TSx p ey e r 


Sod e ty 


Full Sap? p L e 


I15D 


$-24B 


$-1 44 


$6 


Ap p L 1 ca n t 8 


23 2 


-3 7 


4 


236 



91 



-3 71 



-226 



-135 



SOURCE: MORC 



cetcutatlonc from VACIS and other program activity date; 

information systems data and f Ideal records; ettendence end fiscal de te from local 
public schools and community cotleges; HDRC worksite survey; bnemptoymeht Insurance 
earnings and pa y me n t s r eco r ds 5 AFDC payments records; ESP end u.ipartment of Social 
Services fiscal records? HDRC study of ESP staff time aUocetlon; and published end 
unpubllahed data on tax rates, employee frinae benefits, and 
AFDCr Medicaid^ Food Stamps, and Unemployment Insurence. 



I'TPA 



adm 1n1 Et reti ve costs foi 



NOTEF: Within each pa rspectl v a, positive numbers Indlrate gains to thet group 
and negative numbers Ind'cata tosses. 



Results are expressed In 1 984 dolUrs. The f,ai semple Includea 2138 
experlnantals aid 1044 controls (1285 ^.j; pi 1 c^rt ^ , 1897 ruclplants). 



Estimates reflect Bttspnotlvo asbu^iiptl ons ^bo^.c lite change In progrem 

affects after deta collection andea TSr. n rst nunl^^r of each renge aesames thet 
program affects dectlna by 22 percent p^r yeor 6u U.^ the ex t r opo I e 1 1 o n period; the 
second number assumes no decey or Increase. 
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1. In Virginia and in this report, AFDC refers t vi:o ; -ogram for 
fainilies headed by_a sisngle parent, usually a woman. The state 
does not have ah AFDC-U program for *-wo- parent f lilies in which 
the principal earner is unemployed and In which the majority of 
cases are headed by married men. Therefore, the AFDC welfare 
population in Virginia is primarily Konen, 

2. Congress authorized creation of the WIN Program in 1957 as an 
employment and training program fdi? AFDC recipients,. The program 
was jointly administered by the U.S. Department of Labor and the 
then U.S. Department of Health, Education and welfare (now Health 
and Human Services). Initially conceived as a voluntary program to 
provide support^ training, education and counseling services^ the 
program has required since 1971 thst (with certain exceptions) 
applicants tc* and recipients of AFDC register with WIN in order to 
receive AFDC benefits. (In the ter: Inology of the welfare system^ 
these indtviduais required to register with WIN are often called 
"WIN-mandatory,") Further, once on AFDC, a WIN registrant who is 
found employable — • and for_whom adequate social services such as 
child, care are provided — can be required to accept an offered job 
or placement or be sanctioned: i.e., have his/her AFDC grant 
reduced for a specified period of time, the emphasis of the pro- 
gram, has shifted since its inception from longer-term employability 
development and training to immediate job placement* 

3. AFDC recipients are judged WIN-mandatory unless they meet one of a 
number of condi*^ions conferring an exemption from the program^ As 
described in the WIN Handbook, these are individualiS who are: 

1. under 16 years old 

2. enrolled full-time in school anci ^nder 21 years 

3- sick, as detennihed by the income maintenance unit 

^4. incapacitated, as determined by the income maintenance unit 

5. 65 years old or older 

6. living in a remote area: located two hours or more away from a 
WIN office 

7. a caretaker -^f a sick person 

8. a mother of a chil^ under six years of age 

(In Virginia, however, the age limit has been_dropped to five years 
of ag^, because the WIN Hand bo jk allows WIN-mandatory status to be 
conferred cn parents who are ava> fvcn their children for more than 
brief or infrequent periods, and the state mandates kindergarten 
attendance^ which puts parents in this position.) 

4. One reason DSS planners wante^^ a statewide program was their 
practice \n the past of operating initiatives on this scale. A 
second reason was that, in Virginia, s Vork Pule requires that in 
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the absiBnce of the WIN Program, all AFDC applicants and recipients 
meeting the criteria for WIN-mandatory status register with the 
State Employment Cbmmis^ for work, DSS officials preferred 
introducing the WIN Demonstration Program into all of the local 
agencies to this option. For a more detailed discussion of the ESP 
background, see Price, 1985, 

5. Because the research design excluded men from the study sample, 
this report often refers to ESP enrol lees as Women. (See Chapter 2 
for a discussion of eligibility requirements for the sample.) 

6. The requirement to operate work experience programs was articulated 
in state legislation of the 1970s. Education and training 

^^^^ added to the progrr i model in response to strong 
sentiment voiced by several pubMc officials at legislative 
hearings on ESP. 

7. At this point, they became ESP enrollees." Throughout this 
report, this term refers to all if. viduals registered with ESP. 

8. This _ report refers throughout tc "applicants'^ and "recipients," 

i?^icate w. ther people Were applying for 
welfare or receiving it at the point of entry into the research 
sample. In fact, many people referred to as "applicants" began 
receiving welfare shortly after they Joined the sample. 

9. De' -iiis_ of the criteria for selecting agencies to participate in 
th^ research are contained_ in the evaluation proposal. They 
iir: lude:_ willingness to participate in an evaluation based oh an 
e.cperimentai design; geographiGal location and population density 
(urban/rurai|^ the WIN-mandatory AFDC caseloadj service or 
^^^^ ^^^^•^^^^^i^y__^oT_ job search and work experience compo- 
nehts; and ease of access to Key data on program participation^ 
AFDC payments and^ information on the cost of providing ESP 
services. Interest in the research was a very important factor, 
vtie state thvit-^c- a ntmber of agencies to participate, and among 
these agencies, v^iilingness to join the evaluation was a primary 
consideration. 

10. These two research groups were later combined. See Chapter 2. 



CHAPTER 2 

1. See Footnote 3 in Chapter 1. 

2. WlN-mandatory £>tatus is not conferred on mothers of children under 
six years of age who personally _provide full-time care with only 
brief and infrequant absences from their children. However, in 
Virginia, attendance at kindergarten is mandatory, and state 
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regulations specif^ that mothers of five-year-olds should be 
dbhsidered WIN-mandatory. 

3. Bane and Ellwood, 1983. 

H. Over 41 j)ercent of the members of the control group in this study 
were employed at some point during the first year following random 
assignment and about the same percentage were off welfare by the 
end of this period, 

5. In this study ^ the im^ ac', -''^alysis compared all members of the 
control group tc rill me«i^er.? of the combined experimental/control 
groups— bof'i ^ cgra;:: .^drticlpants and _ nbnparticipahts. If 
participants a. lad ueen sCu/Jied, it would have been necessary; 

j^'^^r ^^^^ control group, a similar subgroup of 

individuals who would have participated if the program had been 
available to them. This is virtually impossible, since so many 
unmeasured characteristics, such as motivation and situational 
circumstances^ are usually related to participation. Thio, the 
r®?®^'*?^ ^J^^^y^i^ cannot break out the participant groups for 
separate analysis. 

6. The dependent variables consisted of a series of quarterly and 
cumuiative_measures of einployment, earnings anrl liFDC receipt. The 
independent^ variables consisted of one represeiiting membership in 
the_ ex_perimentai group, along with a series of control variables 
representing demographic characteristics that were likely to 
?D?^°^1^^_®'y^l^y°^®"^» earnings, or welfare receipt of . ' rimentals 
and controis alike. the independent variables were entered into 
the regression equations simultaneously. 



This procedure permitted a reliable estimation of the experi- 
mental/control differences in the values of impact measures, that 
were directly associated with exposure of the members of the 
experimental group to ESP. The regression coefficient associated 
with the variable for membership in the experimental group 
r^P'^^?^^^?®^ >^he experimental/Gdntrol diff^' cnce. For each measure 
of program impact, a two-tai.led t- ?r*rormed to determine 

whether experiB.intal/ control di^ were statistically 

significant. 

?• I?_?ciny other "^ork/weli c-re progratjS, regiiPtrat^on includes assess- 
ment for seryl-Cv:^* ESP assesses enroilees at a later point. 

8. Based on computer^generated random numbbrs, the research code- were 
distributed sepa^-ately for e^ach office in order to maintain a . 3pre- 
sentative distribution of individuals in each of the three resiMrcli 
groups. J©Re kept i^^^ individua?.s in order to avoid 
assigning people more than once. 

9. Henry County also phased In enroilees quickly: 22 percent of its 
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caseload was randomly assigned during these months^ but that 
percentage represents only 19 women. 

10; See Footnote 2. 

11. table 2.2 iridicates that the percentage of individuals screened but 
by agency staff members varied fro^^j 30 percent in the_rij.ral areas 
to 5^ percent in Chesapeake. Tlvs^^e differences reflect agency 
variation in ^ the proportion of : ^ople excluded or different 
reasonsi Table A. particoJ arly _ shows th< large proportion of 
women who w_ere HIN-mandatory whose youngest child was under six. 
This group mostly contained people already in training or triothers 
with five-year-old children, who were likely to be in kindergarten. 

12. The Job Search-Work Experience group ^ have a somewhat lower 
emplbynent rate and earnings average far _the year prior to random 
assigrjneht than did the other_ two _experimental groups. The 
regre:3sibn-adjuj3tiaent procedure employed in this study, aside from 
increasing the precision of estimtion, controls for the few small 
but statistically signifieaht_ differences ^■'.mong research groups 
that generally occur with random assignment. Baseline character- 
istics used for adjustment Include prior _wqrk_ history ^ education, 
and pribr_ welfare receipt. It is a.iways possible, of course^ that 
individuals in the re3earch groups may vary according to any number 
of unmeasured characteristics (such as motivation or adaptability 
to new situations). However, the random assignment process gener- 
ally results in an unbiased distribution of individuals who vary 
according tb these unmeasured characteristics. 

13* It is alsb interesting to note that, among^ controls, 5^ percent cf 
the_ appiioants, but only 32^-5 percent of the- recipients found 
emplbynent from the second through the fourth quarters following 
rahdbm_ assignment. l^^early §8 percent of the recipient controls, 
but bnly 68 percent of the applicant controls^ received welfare at 
some time during the 12 months following random assignment. 

14. A quality cbhtrol ch'^ck revealed that data on welfare payments in 
VACIS underestimated slightly (on average by $1,883 the actual 
value of enrollees* welfare grants. See Appendix B on Quality 
Control. Since this discrepancy is within the range usually founc' 
in quality control checks bf state infonnatibn systems in MDRG 
studies of WIN Dembnstratioh prbjects, the payments data were not 
modified fbr the analysis in this report. 

VACIS did hot recbrd deregistratiori for those individuals who did 
nbt receive ahj AFDC payments for nine months following random 
assignment. Hbst bf these People were applicants who were 
registered during their eligibility interviews and subsequently did 
not have their welfare grants approved: i.e., about 32 percent. 
HDRC classified these people as deregistered in order to avoid 
understating program participation. 
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15. The AFDC payments data submitted by Fairfax Gburity were ndt as 
accurate as the VACIS_ data from the other agencies^ (See Appendix 
B,) Nevertheless^ HDRC concluded that they were accurate enough to 
warrant their Ihcluslbh In the analysis, 

16. Weighting was used to avoid uhder-reportlhg Fairfax enrol lees in 
the process analysis. 

17. Since a quality control _check bf_ Quarterly data oh earnings and 
monthly data on benefits from the UI System found their quality was 
high, these data were used without modification. (See Appendix B.) 
Additionally^ a check T^as done to determlhe the proportion of 
Fairfax County sample members who were employed in Washington^ D.C. 
Very few individuals were; thus, earnings data came exclusively 
from Virginia. 

In _addition to the UI System, VACIS also supplies job placement 
data, but it is much less complete. Employment data for VACIS are 
^?^T'^?®P9r^?^l::^"l^®^'^<5].lee in the experimental group who found a 
job was supposed to inform the Employment Services Worker who would 
record the change of status and date in VACIS. A cbmparisbn bf_jbb 
placement rates included in VACIS and rates calculated from 
Unempioym^en^_Insurajice rec^ found that VACI_S included bhljr half 
the employment recorded in the UI system. Therefbre, VACIS job 
placement data were not used for the impact and benefit-cbst 
analyses, and are only reported as performance indicatbrs in 
Chapter 3; 



i; The ESP model did not require local agiBncies tb directly prbvide 
education and training services. In fact, agencies were ericbur- 
aged to use community resources. 

2. SeeChapter 2 _for a description of the registratibn process. It is 
important- to note that registration was carried out by intake br 
continuing-eiigibility workers who did not make assessments fbr 
possible assignment to ESP services. For assessnient, registrants 
were referred to ESP Workers, who scheduled them for interviews. 

3* Only _Newport News gav^e^ enrollees a series of vocational tests 
as part of the assessment interview; The test results did not 
general ly_ prove to be very useful in either guiding enrollees or 
helping workers assign them to ESP services; 



Data collected for the benefit-cost study indicate that partici- 
pants in workshops in Hampton and Chesapeake participated, on 
average, for 1G days. 
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5. Enrbileeis who were not in compliance with program requirements and 
were judged hot to have_ good cause for their actions were referred 
to their eligibility worker_f or sanctioning — i.e., a reduction of 
the grant over three successive months. In^ slngl e- parent AFDC 
case, only the adult* s portion-of the grant was not paid during the 
sanctioning period. During the three months after the referral for 
sanctioning, the ESP case was closed and the enrollee was con- 
sidered to be dereglstered. 

6. Data oh assignment to cbmpohehts in Fairfax County werB collected 
only for the first assignment to each activity. Therefore, the 
prbpbrtibh of experimehtals reassigned to Job search does hot 
include experimentals in Fairfax whose additional assignments to 
job search were never recorded. Ongoing participation in this 
activity may thus be understated. 

7. During the period under study, Newport News provided the majority 
c>f its group job search services through a three-day orientation 
which preceded a mbhth-lbng individual search. The three-day 
orientation has since been replaced by a four-week workshop to 
which enrbllees heeding mere intensive preparrtibh and support are 
assigned. This hew activity also differs from the workshop 
described in the interim repbrt In that telephones are how 
available fbr calling employers and participation is required. 

8. it is nbtevbrthy that staff in twb bf the agencies (Hampton and 
Fairfax) indicated that subsequent tb the clbse of the evaluation 
period, their agencies had begun tb develop ways bf strengthen- 
ing individual job search without Ibsing the flexibility that it 
offered to enrbllees and staff. Before enrbllees undertake an 
individual search, the agencies hbw provide a small group briehta- 
tion lasting three days. Hamptbh asks participants to come back 
for a group session at the midpbint of their individual search tb 
share their experiences and receive feedback and further guidance 
from staff. 

9. Chesapeake operated a GED preparatibh class available to all DSS 
clients. Classes were scheduled in six-week cycles and could 
accommodate 15 participants. 



CHAPTER it 

1. Out of the full sample of 3 , 18^1 individualSi^ twb cbhtrbls did hbt 
have Social Security numbers and were therefbre e;:cluded frbm the 
impact analysis. 

2. If it could be presumed that nbhparticipatihg experimentals are 
identical tb particiF^ahts _a_hd tb cbhtrbls in measured and 
unmeasured characteristics affecting outcbmes^ then it would be 
appropriate tb exclude hbhparticipants frbm the analysis. Hbwever, 
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no such prestimptlbn is warranted. Those who do hot participate in 
an enjployinent program may difrer systematically frbu those who do 
participate and from cbntrbls. Fbr example ^ hbnparticipants may be 
less motivated to work br less educated. Oh _ the other haridi^ 
.^^HP^rtioipants in an empioyment program may be thbse whb can find 
work on their own without any help frbm the prbgram. In any case^ 
?:r__?^?P^'^ticipants and^ differ systematically, then 

participants and controls differ systematically, and cbmparisbns 
which exclude nonparticipants would be tainted with selectibn bias. 

For this reason, no distinction was made between nonparticipating 
and participating e^xperime in calculating any of the impacts 

reported in this chapters 

3. As_ part of thia study,__ESP _staf f identifi schools and 

community colleges that were most likely to be attended (18 in_all) 
by ESP enrolieesi i^^^se institutions^ were later given a list of 
experimental s and_ controls and asked_ whether or not they had 
provided education and training seinrices to jiny of those people 
during the study period. This type of information was collected 
for a stratified random subsample of 36G experimehtais and 
controls. 



For rates of participation in JTPA, a list of 1,556 randomly 
selected experimehtais and cdhtrols was submitted to the 6overnor*s 
Emplbymeht and Training Division, which _ used statewide JTPA 
information system data to icle^itify which persons on the list had 
been served by JTPA agencies during the study period. 

i|. The prbpbrtibh of control applicants who received welfare can be 
found in Table J*. Si while that of cbhtrQl recipients _ is seen in 
Table 2*. 5. The fact that not all recipients were on the rolls 
could stem from several reasons, among them: errors or lags in 
reporting welfare payments or keypunch errors while inputting 
payments records into VACISi 

5. Appendix Tables F.2 through F.U show that for the full sample as 
well as for the applicant subgroup, the differences in Impacts 
between the two experimental groups were, in fact, hot 
statistically significant. the one exception was that for the 
recipients^ welfare savings were statistically significant s 
greater for the All ESP Services group than for the Job Search/Work 
Experience group. in contrast, the employment impacts of the two 
sequences did not differ for recipients or for applicants to a 
statistically significant degree. Thus, because the two groups had 
similar experiences, both in terms of service receipt and prbgram 
butcbmes^ the findings in this chapter are based on data from the 
combined experimental groups compared to data for the entire 
cbhtrbl group. 

6. For example, for those who entered the sample earliest^ in August 
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1983 1 the first quarter for AFDC outcomes is the period from August 
through October 1983. However, the first quarter for employment 
outcomes f or _ that same group is the period from July througf: 
September ig83._ For those who entered the sample last^ in Septem- 
ber ig8it, the first quarter _f or AFDC outcomes is the period from 
September through November ig84| while the first quarter for employ- 
ment dutcdmes is the period from July through September 1984. 



7. In the technical literature^ the niethbd used to calculate each 
impact and average dutcbme reported in this chapter is known aa 
completely randomized analysis of cbvariahce, See^ for examplei 
Edwards, 1985- 

8. That is, if the effect df the prdgram dn its target population 
really were zerOj a difference as large as that observed in the 
case of_ this sample would occur by chance less than 10 percent of 
the time. Thus, a statistically significant sample impact leads to 
the_ inference that a program has an effect dn its target popula- 
tion. 

9. Since this study is focused on experimental-cdntrdl differehcesi rib 

statistical significance, were perfbrmed bn differences in 
program outcomes among specific subgroups withiri the experimental 
or-control research grdups. Specifically, rid direct cbmparisbris of 
^X?''^^ P^^Si^aiS outcomes were made betweeri urbari arid rural 
experimentals, urban and rural controls, arid sd dri. 

10. Regressions for_ these subgroup analyses were run dri the full sample 
??^56z__i'^teraGtion terms, including treatment status arid 
characteristiG_s of the particular subgroup beirig studied. A 
^^^~^^y_:^'^^^y^^^ ^^^.c^^M'^iance was used td calculate the impacts. 
Treatment s_tatus was always the first factdr, arid each subgrdup 
characteristio, in turn, was used as the secdriji factdr, arid 
interacted with the first, or treatment, status. Thus^ treatment 
status interaGted with one subgrdup dimerisidri at a time, rather 
than with all dimensions simultaneously. 



CHAPTER 



1. ^*wo of these^ evaiu^ are especially relevant because they 

assess programs that served AFDC recipients. See the evaluation of 
the National Supported Work _Demonstratiqa by et al. , 1981 j 

and the evaluation of the Employment Opportunity Pilot Project by 
Long et al., I983. Many of the_ techniques developed in these 
studies were used In the benefiJtj-jJost analysis of ESP and of other 
state programs in MDRe»s_ Demonstration of_ State Work/Welfare 
initiatives^ For addiiionai information on the use of these 
procedures in JffiRC's evaluation of a job search and work experience 
program in California see Long and Knox, 1985. 




2* P^ta collectibn ended ih_ June 1985 fdr earnings and program 
tracking data, in A_ugust igSS fbr Uhemplbymeht Insurance payments, 
and in_ September 1985 for AFDC payments. Data on partlclpation- 
^f^'^^?^^ ^*^sts and on the use of jeducatioh and training programs 
operated by ccmmunity agencies and schools were cdl led ted through 
November 1985, ESP operatiiig costs cover caljhdar year 1984, See 
table 2,6 for the length of fbllbw-up _fpr AFDC pajTments, program 
tracking data, and UI earnings and benefits. 

3. Using microsimuiation techniques, Sneeding estimated the value of 
fringe benefits as 17,9 percent of wages and salaries for workers 
earning less than $10,000 in 1979. See Sneeding, 1981 • 

The amount of _folldw-tip^ the common period ranges 

from four quarters for sample members randomly assigned in August 
and September^ 1983 to none for those randomly assigned in July 
through September 198^1. 

5. Recipients paid a slightly _higher amount of federal taxes than did 
applicants, despite the fact that tlreir^ earnings gains were lower. 
This reflects the fact that applicants, an average, received a 
larger Earned Income Tax Credit ttan recipients, which offset a 
greater portion of the federal taxes they owed, 

6. See Kemper and Long, 1981, and Long _and Knox, 1985 for information 
about the technical aspects of estimating the value of in-program 
output, 

7. See Chapters 2 and 3 and Appendix E for information on the worksite 
survey, 

8. Basing the value of output dh the wages and fringe benefits of 
alternative workers assumes that the compensation employtrs pay 
does in fact represent the empidyees' contributions to output. 

9. The true social value of that output is probably somewhat less than 
the value estimated dn the basis of the cost of alternative labor. 
See Kemper and Ldng, 1981. 

10. Food Stamp regulatldns also allow a deduction for out-of-^pocket 
work-related expenses such as child cardi Hdwever, since accurate 
data on these expenses were not available for this study, the 
estimated program effects dh Fddd Stamps may be somewhat 
underestimated. Data on the average value of Food Stamps for AFDC 
recipients were obtained frdm the Division of Benefit Programs in 
DSS, 

11, Until October 198^, the llSit_ was fdur mdnths. _ Subsequent 
regulations required states to provide nine months of Medicaid to 
former- AFDC recipients who Idst their AFDC eiigibiiity due td the 
termihatibh of the earnings disregard. Hdwever, for the 
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benefit-cost evaluatlbn pf eSP^ the estimated program effects oh 
Medicaid were based on the fbur-mbhth liml t. The analysis could 
not accurately determine^ the prbpbrtibh bf experlmehtals and 
controls who lost their AFDC eligibility due to the termlhatloh bf 
the earnings disregard, 

12. Data on the average value bf Medicaid payments for AFDC recipients 
were obtaijied from the Virginia Department of Medical Assistance 
Services, Fiscal Agent Repbrt for Fiscal Year 1984. 

^? ^" Chapter Jl, the shdrt-tenn, impacts bn AFDC payments were 

statlstically significant for the full sample and for recipients,^ 
^or ^PPlicsints. However, bver_ the longer fbllbw-up peribd 
?9r_^he early enrollees, the impacts for recipients were not sus- 
tained, while, ^or applicants, they became statistically, signifi^ 
cant in the final quarter of fdlldw-up. See Tables U.2 thrbugh 
4.6. 

i4i this approach_ to_ e^stimat and Medicaid administrative cost 

savings differs from the approach used in several previous evalua- 
tions_of state programs in the Demonstration of State Work/Welfare 
Initiatives^^ In those evaluations, administrative_ costs were 
estinaated as a proportion_ of the value of the transfer payments. 
However^ since _ administrative costs are more likely to vary with 
oj* A^™® ^'^a^^ ? c^se remains open than with the benefit 
ievei^__the approach in this evaluation of ESP is probably more 
_ __P^t_^_:^PJ^:^tate and local administrative cbsts for 
Medicaid were obtained from the Virginia Department of Medical 
Assistance Services, Fiscal Agent Report for Fiscal Year 1984 j 
comparable data for AFDC were obtained from the Division of Benefit 
Programs in DSS. _ Federal admlnistr^^ cbst data for Medicaid__and 
AFDC were obtained from the Appendix to the Budget for Fiscal Year 
1984i 

15. For _Fb6d Stamps^ and _ UI, data were not available to permit 
estimation_ of administrative costs in relation to the average 
length of time spent on the caseload. State and local Fbqd_Stamps 
administrative cost data were obtained from the Divisibn of Benefit 
Prbgr^s in_DSS, and federal data were obtained from the Appendix 
to the Budget for Fiscal Year^ Administrative cost data for 

Unemployment Insurance were obtained from the Virginia Employment 
Commission. 

15. Research reported ill Bane and Eiiwood, 1983 found that a national 
sample of AFDC recipients remained continuously on welfare for an 
average of 4.7 years. 

1?. Although, as reported in Chapter 4, _ the program's estimated impacts 
on ear ni:igs and. AFDC -payments during^ this base period were 
generally hot statistically significant, the relatively stable 
pattern of earnings gains and reduced welfare payments during the 
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entire observation period suggests that the base period effects are 
not simply due to chance, 

18. For example, see the evaluation of the National Supported Work 
Demonstration (Masters and Maynard^ 1981) and the evaluation of a 
WIN job search program in Louisville, Kentucky (Wolfhag€?h, I983)i 

19. See Ketron, Inc., I9S0. 

20. A g;eneraiiy accepted range of discount rates is from 3 to 10 
percent. (See Kemper et ai.^,^ 19^11.) Because the time period for 
this study is ahiy five ye^^^ the results of the analysis would 
hot be substantially affected by choice of a higher or lower 
discount rate. 

21. The period from January through December 198^1 was selected as the 
peri6d_bh which to base the estimates of average program costs in 
order to exclude start-up costs (which are not typical of bh-goihg 
operating costs) _ in estimating the average costs of program 
services. Moreover , this ^^^^ the time during which a 
major share of_ the active participants in the research sample took 
part in program activities. 

22. Staff were asked to estimate theproporiicn of their I985 work time 
that was spent oh each major program component. Although it would 
have been preferable to obi&aih the^e estimates for 1984 in order to 
correspond to the period used for riscai- data, the allocation of 
staff time is likely to have been quite similar in both years, 
sihce changes in program operations between 1984 and 1985 were not 
substahtial. 

23. Unit costs for assessmeht, individual job search, group job search, 
ESP educatioh a_hd_ train^ir^,^ experienGe in two counties 
where this component, was subcontracted (Newport News and Honpton) 
were each estimated as a fixed cost, that is, as a cost per person 
entering a program statjis. _ (For example, the average unit cost of 
assessment was jdetermined^ by dividing the total expenditures for 
assessment by the number of persons ever assessed. ) The unit cost 
of reassessment was estimated as a variable cost^ which differs for 
ihdividuals depehdinqg on the number of times they were reassessed. 
Similarly, the unit cost for work experiehce in agencies- other than 
Hamptbh and Newport News was estimated as a variable cost, which 
differed for individuals accordihg to their length of stay in work 
experiencei Data oh the humber of hours spent in work experience 
were obtained from ESP ease records for a stratified random sub-- 
sample of 111 experimentals assigned to work experience. In some 
cases (e.g., for individual Job search) a variable cost approach 
may have beeh more appropriate than a fixed cost approach but was 
not feasible due to data iimitatibhs. 

24. These regression-adjusted estimates were almost identical to the 
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non-regressibh-adjusted differences In partlcipatlbn rates between 
experiinentals and cbritrbls. 

25. Thus, the analysis, makes ah assumption about the similarity of a 
certain subgroup of both the later enrollees (who entered ESP from 
April through September 15845 and the earlier enrollees (who 
entered from August 1983 through Harch 1984): that those still 
enrdlled in the program in the ninth month after random assignment 
in both groups have a similar probability of entering an ESP 
component for the first time in subsequent months. 

26. Records of purchase of service payments fbr both Title XX and ESP 
expenditures were examined for a stratified random subcample of 
1,758 experimentals and contrdls. 

27. Because^^ata on these activities were available thrbugh November 
1985, and because the overall participatibh rates were, low, 
participation rates for these activities were hbt extrapolated. 
For_ further information on this special study, see fbbthbte 3 of 
Chapter 4. 

28. Public school cost data were obtained frbni the Vbcatibhal Research 
and ftduit Services _division of the Virginia Department bf 
Educatioh^_^ The Virginia Community College System Of fice provided 
data on-community college costs. JTPA cost data were obtained froin 
the Governor's Employment and Training Division of the Cbmmbhwealth 
of„yirg±nia._ Although the participatidn available from JTPA 
identified specific categories of activitiies, fiscal data for the 
same categories were not available. TherefNDre, it was bhly 
possible to compute an average unit cost per JTPA activity, rather 
than separate unit costs for each category of activity. 

29. However^ because the experimental-cdntrbl difference in the overall 
rate of_partJ.cipatio_n in educatidn and training activities was 
small^_inaccuraeies in the estimates of unit costs would hbt have a 
substantial effect on the overall results of the behefit-cbst 
analysis. 

30. Several prior MDRe_ evaluations of state prdg-ams in the 
Demonstration or State Work^^^^ Initiatives fdcused mbre bh the 
taxpayer and sociatajt perspectives than the budgetary perspective. 
Since the __budgetary__perspectiv^^ has sttracted cdhsiderable 
attention from policymakers recently, this benefit-cdst analysis bf 
ESP emphasizes this perspective more. 

31- The term "taxpayers" as used- here refers to everydne in society 
excep)t members of the welfare_ sample included in this study. 
However, It is important to note that some experimentals are 
taxpayers. As discussed_ in this chapter, one effect df ESP was to 
Increase the amount of taxes they paid; 
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32. This analysis assumes that the earnings of taxpayers are not 
reduced to the same extent that earnings increase for experlmehtais 
^- a possibility if some taxj)ayers are displaced from their Jobs 
due to increased employment of experimehtals. 
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